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Abstract 

As  an  Air  Force  Chief  of  Staff  endorsed  topic,  Air  Force  Studies  and  Analyses 
Agency  (AFSAA)  requested  an  effective  and  efficient  way  to  reduce  the  variance  in 
analysis  results  from  THUNDER.  THUNDER  is  a  large-scale  discrete  event  simulation 
of  campaign-level  military  operations  and  is  used  to  examine  issues  involving  the  utility 
and  effectiveness  of  air  and  space  power  in  a  theater-level,  joint  warfare  context.  Given 
the  large  number  of  stochastic  components  within  THUNDER,  results  are  produced  with 
highly  variable  measures  of  effectiveness  (MOEs),  causing  difficulties  in  evaluating 
alternative  force  structures,  weapon  systems,  etc. 

This  work  responds  to  AFSAA’s  request  by  examining  the  application  of 
Common  Random  Numbers  (CRN),  Antithetic  Variates  (AV),  Control  Variates  (CV), 
and  a  combination  of  AVs  and  CVs  and  a  combination  of  CRNs  and  CVs.  The 
difference  between  the  nonnal  and  variance  reduction  technique  halfwidths  for  95% 
confidence  intervals  were  examined.  Analysis  of  the  correlation  between  MOEs  and  the 
random  inputs  in  the  C  V  technique  gave  insight  into  the  workings  of  THUNDER.  A 
new,  state  of  the  art  combined  multiple  recursive  generator  was  incorporated  into 
THUNDER  to  synchronize  the  random  inputs  for  CRN  and  AV.  The  result  is 
methodology  for  implementing  all  four  variance  reduction  techniques. 
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FEASIBILITY  STUDY  OF  VARIANCE  REDUCTION 


IN  THE  THUNDER  CAMPAIGN-LEVEL  MODEL 


1.  Introduction 


1.1.  Background 

Over  the  years,  the  Department  of  Defense  (DoD)  has  been  tasked  to  continue  to 
be  the  leader  in  military  strength  and  technology,  with  less  and  less  funding.  One  way 
the  DoD  has  done  this  is  through  the  use  of  stochastic  combat  simulation.  Stochastic 
combat  simulation  is  a  way  to  model  how  the  input  parameters  affect  a  specified  measure 
of  effectiveness  (MOE)  in  a  combat  situation.  To  do  this,  an  analyst  uses  simulation 
software  to  construct  a  combat  simulation  by  assigning  probabilities  to  weapons  systems 
and  logical  flow  in  order  to  detennine  the  interaction  of  the  simulated  combat  forces. 

The  results  from  a  stochastic  combat  simulation  are  point  estimates  because  they  rely  on 
a  series  of  pseudo-random  numbers.  These  pseudo-random  numbers  are  used  to  model 
the  randomness  of  a  system.  This  randomness  can  greatly  affect  the  simulation  results, 
leading  to  a  range  of  estimates. 

There  are  many  known  ways  to  reduce  the  variance  in  the  MOEs.  The  most 
common  method  is  to  increase  the  number  of  replications  completed.  This  method  can 
prove  to  be  inefficient  due  to  the  size  of  the  simulation.  Even  with  the  speed  of  today’s 
computers,  some  of  the  larger  simulations  can  take  hours  or  even  days  to  complete  one 
replication.  Therefore,  an  increase  in  replications  may  be  unacceptable.  Other  methods 
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of  reducing  the  variance  are  called  variance  reduction  techniques  (VRTs).  These 
techniques  use  knowledge  of  the  workings  of  computer  simulations  and  mathematical 
techniques  to  reduce  the  variance  without  increasing  the  number  of  replications  (20:581). 

The  DoD  uses  a  variety  of  large,  stochastic  combat  simulations  to  conduct  various 
analyses  of  interest.  These  interests  vary  from  acquisition  to  planning  to  training.  One  of 
the  large,  stochastic  combat  simulations  used  by  the  Air  Force  (AF)  is  THUNDER. 
THUNDER  is  a  stochastic,  two-sided,  analytical  simulation  of  campaign-level  military 
operations  developed  in  the  1980s  under  the  guidance  of  the  Air  Force  Studies  and 
Analyses  Agency  (AFSAA).  The  simulation  was  designed  and  built  expressly  to 
examine  issues  involving  the  utility  and  effectiveness  of  air  and  space  power  in  a  theater- 
level,  joint  warfare  context. 

1.2.  Problem  Statement 

Given  the  large  number  of  stochastic  components  within  THUNDER,  the  model 
often  produces  MOEs  with  significant  variance.  This  variance  often  causes  difficulties  in 
evaluating  alternative  force  structures,  weapon  systems,  etc.  With  time  sensitive  analysis 
and  long  replication  times,  AFSAA  is  looking  for  an  effective  and  efficient  way  to  reduce 
the  variance  in  the  results  from  the  THUNDER  campaign-level  model. 

1.3.  Research  Objective 

The  objective  of  this  research  will  be  to  study  the  use  of  variance  reduction 
techniques  on  THUNDER.  The  study  of  existing  variance  reduction  techniques  will 
determine  which,  if  any,  are  effective  and  efficient. 
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1.4.  Research  Focus 


This  research  examines  the  use  of  several  classic  VRTs.  The  primary  focus  of 
this  research  identified  potential  control  variates  (CV),  within  the  model,  that  can  be  used 
to  reduce  the  variance  in  the  output.  Then  using  these  potential  control  variates, 
determine  how  much  each  potential  control  variate  influences  the  MOE  variance.  The 
areas  are  also  checked  to  find  out  if  there  is  any  other  information  that  can  be  gleaned 
from  the  relationship  between  the  identified  control  and  the  simulation  MOE. 

A  secondary  focus  of  this  research  examines  the  use  of  two  other  classic  VRTs. 
The  second  VRT  applied  is  common  random  numbers  (CRN).  CRN  is  probably  the 
second  most  popular  way  of  reducing  variance,  following  increasing  the  number  of 
replications.  It  is  primarily  used  when  trying  to  identify  differences  between  simulation 
configurations.  The  basic  idea  is  to  reduce  noise,  generated  by  the  random  inputs,  in  the 
variation  of  the  system.  The  third  VRT  applied  is  antithetic  variates  (AV).  AV  attempts 
balance  out  any  radically  low  or  high  random  input.  Also,  the  application  of  combining 
AV  with  CV  is  reviewed. 

1.5.  Methodology 

In  the  control  variate  technique,  the  primary  focus  is  in  the  generation  of  random 
variates  throughout  a  simulation.  Initially,  the  random  variate  averages  are  generated  and 
stored.  The  differences  in  the  simulated  mean  and  expected  value  are  regressed  against 
the  simulation  MOEs.  This  identifies  which  random  variates  are  influential  to  the 
selected  MOEs. 

To  use  the  CRN  technique,  the  pseudo-random  number  calls  need  to  have  their 
own  random  number  stream.  THUNDER  already  has  a  way  to  assign  different  random 
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number  streams.  The  problem  with  THUNDERs  current  RNG  is  that  there  are  only  10 
streams  to  choose  from  and  245  different  random  number  calls.  Therefore,  a  new  random 
number  generator  is  needed  to  allow  for  all  the  random  number  calls  to  have  its  own 
stream.  In  this  research,  the  random  number  generator  suggested  by  Pierre  L’Ecuyer, 
Richard  Simard,  E.  Jack  Chen,  and  W.  David  Kelton  (18)  was  added  to  THUNDER. 

1.6.  Aeronautical  Systems  Center  Major  Shared  Resource  Center 

Simulation  replications  for  this  research  were  performed  with  cooperation  with 
the  Aeronautical  Systems  Center  (ASC)  Majro  Shared  Resource  Center  (MSRC).  The 
ASC  MSRC  is  a  computational  science  facility.  It  supports  various  DoD  research 
initiatives  including  development,  test,  and  evaluation.  As  part  of  the  DoD's  High 
Performance  Computing  (HPC)  Modernization  Program,  the  ASC  MSRC  uses  high 
performance  computing  research  and  visualization.  Access  to  the  ASC  MSRC  HPC 
resources  was  facilitated  by  use  of  the  Divide  and  Conquer  (DAC)  system.  The  DAC 
system  is  fronted  by  a  web  interface  that  assists  in  configuring  experiments  and  has  direct 
access  to  HPC  resources. 

1.7.  Assumptions  and  Limitations 

The  major  limitation  of  this  research  occurs  due  to  the  classification  of  the 
THUNDER  model  databases.  Since  most  THUNDER  model  databases  represent  a  real 
battle  plan,  implement  new  military  technology,  or  contain  specific  data  on  current 
military  systems,  most  all  THUNDER  model  databases  are  classified  to  some  level. 
Therefore,  the  only  unclassified  model  databases  available  are  the  three  notional  model 
databases  distributed  with  THUNDER.  The  model  database  used  in  this  research  is  the 
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Middle  East  (ME)  scenario.  Assuming  the  VRT  affects  THUNDER  and  does  not  rely  on 
the  scenario,  the  VRT  can  be  used  on  any  other  THUNDER  scenario  study. 

1.8.  Overview 

There  are  five  chapters  in  this  research.  Chapter  1,  the  Introduction,  contains 
background  information,  problem  statement,  research  objective,  a  synopsis  of  the 
methodology,  and  the  assumptions  and  limitations  of  the  research.  Chapter  2,  the 
Literature  Review,  contains  descriptions  of  THUNDER,  other  THUNDER  work,  CVs, 
CRNs,  AVs,  other  VRTs,  and  analysis  techniques.  Chapter  3,  the  Methodology, 
discusses  how  CVs,  CRNs,  and  AVs  are  applied  in  THUNDER.  Chapter  4,  Results  and 
Analysis,  presents  the  output  of  the  selected  ME  scenario  MOEs  and  the  results  from  the 
application  of  CVs,  CRNs,  and  AVs.  Chapter  5,  Conclusions,  is  a  look  at  the  analysis 
done  in  the  previous  chapter  with  recommendations  for  future  work  in  the  application  of 
VRTs  and  THUNDER. 
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2.  Literature  Review 


This  chapter  contains  background  information  on  software,  models,  techniques 
and  services  used  in  completing  this  research.  It  begins  with  a  discussion  on  combat 
simulations,  specializing  in  the  THUNDER  campaign  level  combat  simulation.  It 
continues  with  a  description  of  THUNDER’S  computer  language,  SIMSCRIPT  II. 5. 

Next,  there  is  a  lengthy  section  on  variance  reduction  techniques.  This  section  reviews 
Control  Variates,  Common  Random  Numbers,  and  Antithetic  Variates  in  depth,  while 
giving  a  brief  overview  of  other  techniques.  To  close  this  section,  there  is  a  review  of 
other  variance  reduction  studies.  Next,  the  random  number  generator  added  to 
THUNDER  for  use  with  CRN  is  described.  Following  this,  is  a  short  description  of  the 
ASC  MSRC’s  Divide  and  Conquer  system  used  to  replicate  the  runs.  This  chapter  ends 
with  an  overview  of  variance  and  confidence  intervals. 

2.1.  Combat  Simulation 

“Simulation  of  large-scale  combat  models  contribute  useful  insights  for  many 
military  decision  problems. ”(14: 1-1)  There  are  many  reasons  why  a  system  is  modeled 
by  computer  simulation.  For  instance,  a  system  may  be  too  difficult  or  expensive  to  alter, 
the  system  may  not  exist,  or  even  unacceptable  to  simulate  in  real  life.  In  combat 
simulations,  the  latter  is  probably  the  case.  It  would  be  unethical  to  start  real  combat 
merely  to  test  a  new  weapon  system.  The  goal  in  the  design  of  combat  models  is  to 
represent  warfare  as  accurately  as  possible,  because  results  from  combat  simulation  can 
guide  national  defense  and  other  policy  decisions. 
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There  are  many  different  levels  of  combat  modeling.  They  range  from 
engineering  to  campaign  level  simulations,  see  Figure  2. 1  (26).  A  campaign  level 
simulation  encompasses  a  whole  campaign  and  covers  many  different  elements.  The 
sheer  volume  of  entities  would  make  it  difficult  to  simulate  each  entity  interaction. 
Therefore,  they  are  performed  with  low  resolution  and  aggregation.  This  aggregation 
means  that  each  entity  is  not  simulated  but  is  replaced  by  probabilities  and  percentages. 
This  allows  a  large  campaign  level  simulation  to  run  in  an  acceptable  time  frame,  at  the 
cost  of  losing  output  resolution. 


A 


Increasing 
Aggregation  Campaig 


Higher 
Resolution 


Mission 


Engagement 


Engineering 


v 


Figure  2.1:  Combat  Modeling  Hierarchy  (26) 


2.2.  THUNDER 


This  research  is  a  study  on  THUNDER,  a  large  stochastic,  two-sided,  constructive 
simulation  that  has  been  operational  since  1986.  It  was  developed  to  examine  the  utility 
and  effectiveness  of  air  and  space  power  in  at  a  campaign-level  and  models  joint  military 
operations  at  the  campaign-level.  The  Air  Force  Studies  and  Analyses  Agency  (AFSAA) 
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sponsored  the  development  of  THUNDER.  The  following  material  is  obtained  from  the 
THUNDER  web  pages  (13)  and  Volume  One  of  the  THUNDER  documentation  (35). 

THUNDER  is  designed  for  analytical  flexibility,  accessibility,  and  ease  of  use.  It 
operates  in  a  UNIX  environment  and  is  written  in  SIMSCRIPT  II. 5,  a  simulation 
programming  language.  SIMSCRIPT  II. 5  is  ideal  for  the  creation  of  large  scale,  discrete- 
event  simulations.  To  aid  the  analyst,  the  distribution  of  THUNDER  comes  with  in- 
depth  documentation  and  source  code,  allowing  modification  at  the  programming  level. 
The  source  code  of  THUNDER,  alone,  consists  of  around  360,000  lines  of  codes  spread 
across  over  1,500  routines. 

THUNDER  operates  at  multiple  levels  of  detail.  It  models  air  operations  at  a  high 
level  of  detail,  with  the  ability  to  simulate  27  different  air  missions.  While  it  can  simulate 
air  operation  at  a  high  level  of  detail,  it  models  ground  operations  at  a  medium  level  of 
detail,  focusing  on  ground  combat,  logistics,  terrain,  C3I,  and  mobility. 

To  assist  the  user  and  speed  up  replications,  THUNDER  automatically  generates 
its  own  Air  Tasking  Orders  (ATOs)  and  Intelligence  Tasking  Orders  (ITOs),  based  on 
theater-level  apportionment  and  target  priorities.  It  also  uses  a  detenninistic  time  stepped 
ground  operations  model  based  on  the  Center  for  Army  Analysis  Concept  Evaluation 
Model  (CEM)  and  its  Attrition  Calibration  Methodology  (ATCAL). 

THUNDER  can  be  run  in  two  modes:  analytical  and  wargame.  The  analytical 
mode  focuses  on  the  contribution  of  systems,  capabilities,  forces  and  employment 
concepts.  It  is  run  in  a  batch  mode  and  delivers  results  that  can  be  identified  by 
replication  or  as  a  mean  across  replications  with  confidence  intervals.  When  run  in  batch 
mode,  THUNDER  starts  the  following  random  number  stream  where  the  previous  stream 
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stops.  The  wargame  mode  supports  a  user  interaction  in  a  game  like  atmosphere,  where  a 
player  dynamically  influences  the  outcome  of  a  run. 

As  a  member  of  the  Air  Force  Standard  Analysis  Toolkit  (AFSAT),  THUNDER 
supports  senior  decision  makers  (Figure  2.2)  in  matters  of  policy,  acquisition  and 
operations.  Typically,  these  customers  require  precise  and  timely  analysis.  The  analysis 
applications  include:  exploration  of  readiness,  modernization,  force  structure,  and 
sustainability  issues;  comparison  of  alternative  courses  of  action  (ACA);  assessments  of 
evolving  capabilities,  alternative  strategies  and  potential  operational  concepts;  and  the 
facilitation  of  senior  staff  training  through  wargaming. 
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THUNDER  user  base 
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Figure  2.2:  THUNDER  User  Base  (13) 


2.2.1.  Input. 

THUNDER  is  a  data  driven  simulation.  Therefore,  the  individual  model 
parameters  are  held  in  a  data  structure  or  database  and  can  be  changed.  THUNDER  uses 


99  text  files  as  a  model  database.  These  files  contain  all  the  information  relating  to  the 
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entities,  map,  and  course  of  action.  THUNDER  also  allows  the  user  to  use  toggles  to 
tailor  how  a  model  is  run.  The  use  of  the  databases  allows  for  sharing  of  data  and 
minimizing  storage  space.  It  also  aids  in  portability  between  systems  and  even  sharing 
between  users. 

2.2.2.  Output. 

THUNDER  comes  with  three  types  of  post-processors  to  handle  the  hundreds  of 
different  MOEs  available.  The  first  is  a  comprehensive  suite  of  text  post-processors. 
They  provide  the  user  with  a  means  to  extract  information  from  a  simulation. 

THUNDER  uses  an  input  file  to  toggle,  which  postprocessors  are  run  and  which  output 
files  are  requested.  An  example  of  one  of  the  output  files  is  in  Appendix  (A).  The 
second  type  is  a  graphical  post  processor  that  allows  the  user  to  step  through  a  single 
replication  graphically.  It  contains  a  situation  map  and  it  depicts  the  locations  of  military 
units  for  both  sides  over  half  day  increments.  Third,  THUNDER  comes  with  a  post 
processor  that  creates  user  specified  graphs. 

THUNDER  contains  post  processors  that  can  produce  many  different  MOEs.  To 
obtain  the  data  for  a  MOE,  the  user  selects  one  or  more  reports  of  interest  to  produce. 
Three  reports  were  chosen  for  this  project.  The  three  reports  are  CC-4  (FLOT 
Movement),  AW-3  (Aircraft  Loss  Summary),  and  AW-5  (Sortie  Generation).  All  three 
are  contained  in  a  single  output  file  named  reports. std  located  in  the  REPORTS 
subdirectory.  When  running  multiple  replications  of  a  thunder  model,  there  will  be  a 
number  following  the  reports  name.  The  number  references  the  replication  number  from 
which  the  file  was  created.  The  reports. std  file  combines  all  three  reports  and  organizes 
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them  in  time  order,  then  report  name  order,  and  then  order  of  side.  Please  refer  to 
Appendix  (A).  It  contains  the  output  from  the  entire  first  day  of  a  replication. 

The  CC-4  report  is  a  Ground  Combat  Cycle  report.  It  displays  FLOT  movement 
in  square  miles  gained  or  lost  by  command  and  FLOT  location  in  latitudes  and  longitudes 
by  12-hour  increments.  The  AW-3  report  is  an  Air  War  report.  It  contains  losses  and 
sorties  by  aircraft  for  each  day.  The  AW-5  report  also  is  an  Air  War  report.  It  contains 
planned  and  flown  sortie  data  for  each  mission  type  by  base  and  squadron  for  each  day. 

2.2.3.  Previous  THUNDER  Studies. 

THUNDER  is  a  large  stochastic  combat  simulation.  The  size  and  complexity  of 
THUNDER  has  made  it  a  prime  simulation  for  study.  Over  the  years  there  have  been 
efforts  to  analyze  the  inner  workings  of  THUNDER.  Much  of  this  work  has  been  done 
here  at  the  Air  Force  Institute  of  Technology  (AFIT).  Below  is  a  sampling  of  previous 
THUNDER  research  at  AFIT. 

Major  David  A.  Davies  performed  a  sensitivity  analysis  to  command  and  control 
(C2)  inputs  in  THUNDER.  He  employed  design  of  experiments  (DOE)  to  identify 
statistically  significant  variables  to  specified  outputs.  Then  response  surface 
methodology  (RSM)  was  used  to  generate  a  response  surface  of  the  relationships  between 
the  input  and  output  variables.  Major  Davies  also  demonstrated  the  ASC  MSRCs 
capability  to  perform  parallel  processing,  completing  1,560  THUNDER  runs  for  the 
analysis.  Normally  this  would  have  taken  around  3  months,  but  it  was  accomplished  at 
the  ASC  MSRC  in  around  3  days  using  parallel  processing.  (5) 

Captain  Michael  R.  Farmer  studied  how  to  quickly  analyze  the  effects  of 
changing  force  structure  with  THUNDER.  The  modifications  Capt  Farmer  made 
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consisted  of  limiting  the  amount  of  primary  tactical  aircraft.  Capt  Farmer  used  RSM  to 
create  metamodels  of  THUNDER,  which  in  turn  could  be  used  to  provide  a  quick  answer 
to  a  change  in  force  structure.  He  also  used  factor  analysis  (FA)  to  reduce  the 
dimensionality  of  data  sets  and  determine  relationships  between  MOEs.  (8) 

Captain  Steven  L.  Forsythe  applied  RSM  to  a  THUNDER  scenario  in  order  to 
optimize  the  air  apportionment.  Captain  Forsythe  created  a  response  surface  and  used  a 
steepest-gradient  search  to  find  the  optimal  aircraft  apportionment  for  MOEs  such  as 
FLOT  movement  and  attrition.  In  addition,  he  demonstrated  how  RSM  could  provide 
useful  information  as  to  how  the  algorithms  used  by  TAC  THUNDER,  interact  to 
simulate  the  combat  environment.  (9) 

Captain  Brian  M.  Godfrey  compared  five  Air  Expeditionary  Force  (AEF) 
packages.  Captain  Godfrey  used  THUNDER  to  obtain  most  of  the  MOEs  used  in  the 
study.  The  study  results  in  a  methodology  for  modeling  AEF  packages  using  THUNDER 
and  a  methodology  to  analyze  the  results.  (11) 

Major  James  B.  Grier  developed  a  tool  to  capture  the  cost  and  capabilities  of 
alternative  force  structures.  The  tool  uses  FA  and  RSM  to  link  monetary  cost  to  MOEs. 
USAF/XPY  directed  Major  Grier  to  develop  his  tool  using  THUNDER.  His  research 
focused  on  the  Southwest  Asia  scenario  developed  by  HQ  ASC/XR.  (12) 

Captain  Francine  N.  Nelson  conducted  a  validation  assessment  of  THUNDER 
6.5's  Intelligence,  Surveillance,  and  Reconnaissance  (ISR)  Module.  This  study  provides 
an  overview  of  the  ISR  process,  level  of  aggregation,  possible  problem  areas,  and  basis 
for  ISR  improvement  within  THUNDER.  This  study  also  presents  a  sensitivity  analysis 
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of  the  ISR  parameters  relating  to  the  Quality,  Quantity,  and  Timeliness  of  ISR.  DOE  and 
linear  regression  were  used  to  develop  a  response  surface  for  the  sensitivity  analysis.  (27) 

Captain  Timothy  S.  Webb  examined  the  overall  model  variability,  output 
interrelationships,  and  sensitivity  to  input  parameters  in  THUNDER.  The  examination  of 
the  model  variability  focused  on  the  replication  requirements  to  obtain  specified 
measures  of  outcome  (MOOs).  The  application  of  principal  component  analysis  (PCA) 
and  FA  were  used  to  examine  output  interrelationships.  The  input  parameters  were  tested 
for  significance  and  consistency.  (42) 

2.3.  SIMSCRIPT  II.5 

To  understand  the  inner  workings  of  a  simulation,  one  must  learn  the  language  it 
is  programmed  in.  THUNDER  is  programmed  in  CACIs  SIMSCRIPT  II. 5.  According  to 
the  SIMSCRIPT  II. 5  description  web  page  at  CACI,  SIMSCRIPT  II. 5  is  an  easy-to-use 
and  powerful  simulation  language  with  English-like  syntax  (4).  The  English-like  syntax 
facilitates  a  more  intuitive  understanding  of  the  simulation. 

As  a  simulation  language,  it  provides  a  timing  mechanism  and  concurrent  process 
management  in  addition  to  all  the  standard  features  of  a  general-purpose  high-level 
programming  language.  It  has  preprogrammed  discrete  event  simulation  features  like 
entities,  attributes,  sets,  events,  processes  and  resources.  These  added  features  are  key 
elements  of  an  efficient  and  effective  simulation.  SIMSCRIPT  II.  5  also  has  built  in 
graphical  capabilities,  allows  models  to  be  stored  as  a  database,  is  portable  across  a 
variety  of  platforms,  and  can  readily  interface  between  selected  other  programming 
languages. 
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There  are  three  basic  parts  to  the  structure  of  a  SIMSCRIPT  II. 5  program.  They 
are,  the  main  program,  routines,  and  the  preamble.  The  main  program  is  the  essential 
part.  It  consists  of  either  the  whole  program  or  the  starting  and  ending  part  of  a  program 
that  consists  of  calls  to  one  or  more  subroutines.  The  main  program  is  where 
SIMSCRIPT  II. 5  looks  for  the  start  of  the  program.  Routines  are  functions,  subroutines, 
events  or  processes.  They  are  generally  used  to  compartmentalize  the  simulation  code  for 
better  organization  and  ease  of  understanding  the  program  flow,  and  are  activated  by 
either  the  main  program  or  other  routines.  The  preamble  is  where  global  variables  are 
declared  and  constants  are  defined.  THUNDER  was  developed  with  its  own 
nomenclature;  therefore  its  preamble  is  like  a  Rosetta  stone.  It  provides  the  key  to 
translate  words  and  phrases  in  THUNDERs  code  back  into  a  standard  programming 
language. 

SIMSCRIPT  II. 5  is  mainly  used  for  discrete-event  and  combined  discrete- 
event/continuous  simulation  models.  It  is  applied  world  wide  in  building  large-scale 
simulations  and  modeling  applications.  Some  typical  SIMSCRIPT  II. 5  applications  are 
seen  in  Telecommunications,  Network  analysis,  Transportation,  Manufacturing, 

Inventory  control,  Health  care.  Military  operations,  wargaming  and  logistics  planning. 
Some  organizations  that  use  SIMSCRIPT  II. 5  are  the  Federal  Aviation  Administration, 
Saudi  Aramco,  Raytheon,  Rolls  Royce,  the  U.S.  Army  and  the  U.S.  Air  Force.  (4) 

2.4.  Variance  Reduction  Techniques 

A  stochastic  simulation,  such  as  THUNDER,  relies  on  input  that  is  random  and 
therefore  produces  output  that  is  also  random.  This  means,  that  since  there  are  inputs  that 
vary  randomly  from  replication  to  replication,  there  will  also  be  random  variation  in  the 
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output  from  replication  to  replication.  Often  this  variation  is  so  large  it  obscures  the 
estimates  of  the  models  MOEs.  Therefore  analysts  have  developed  techniques  to  reduce 
this  random  variation  in  the  output.  These  techniques  are  called  variance  reduction 
techniques  (VRTs).  There  are  many  different  types  of  VRTs  and  the  effectiveness  of 
each  technique  is  dependent  on  the  particular  model  and  application.  Three  common 
VRTs  are  Control  Variates  (CV),  Common  Random  Numbers  (CRN),  and  Antithetic 
Variates.  CV  and  AV  are  single  model  techniques,  meaning  they  are  applied  to  a  single 
configuration  of  a  simulation,  while  CRN  is  a  multiple  model  technique,  meaning  it  is 
applied  when  two  or  more  configurations  are  compared.  These  VRTs  are  explained  in 
depth  below  and  then  are  followed  by  a  quick  synopsis  of  some  other  VRTs. 

2.4.1.  Control  Variates. 

In  essence,  control  variates  attempt  to  take  advantage  of  any  correlation  between 
random  variates  in  a  simulation  and  the  selected  output.  It  is  best  when  talking  about 
control  variates  to  start  out  with  a  single  response  and  a  single  control.  The  same  theory 
can  then  be  extended  to  multiple  controls.  The  remainder  of  this  section  is  derived  from 
notes  taken  from  Dr.  Bauer.  (1:64-70.5) 

Consider  the  case  were  there  is  only  a  single  control.  Now,  assume  there  is  a 
mean  response  of  interest  from  the  simulation  called  /jy  for  which  Y  is  an  estimator. 
Also,  assume  there  is  another  output  variable,  X,  that  is  correlated  with  the  Y  response 
and  has  an  expected  value  ju  x  that  is  known.  Since  X  is  correlated  with  the  Y  variable,  it 
is  known  as  the  control  variable.  Now  consider  the  controlled  estimator  Y(b)  given  in 
equation  (2.1)  where  b  is  a  constant. 
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r(b)= r -b(x -nx) 


(2.1) 


Note  that  Y(b)  is  an  unbiased  estimator  of  /uY  by  equation  (2.2). 

E[Y(b)]  =  E[Y]-E[b(X-^x)}  (2.2) 

t[l!/i )]-,//;  -Mil,  -//.)-//,  (2.3) 

The  variance  of  Y{b)  is  given  in  equation  (2.4). 

V  ar  (T  (b ))  =  V  ar(7 )  +  Zr  V  ar(X )  -  26Cov(7,  X)  (2.4) 

With  a  little  manipulation  of  equation  (2.4)  it  can  be  shown  that  the  variance  of  Y(b)  is 
smaller  than  the  variance  of  Y  if  equation  (2.5)  holds. 

26Cov(7,X)>  62Var(x)  (2.5) 

In  observing  equation  (2.5),  it  is  apparent  that  if  the  variables  X  and  Y  are  independent, 
then  the  Cov(Y,  Y)  =  0  and  it  follows  that  there  can  be  no  reduction  in  variance  of  Y. 

To  find  the  value  of  b  that  minimizes  equation  (2.4),  the  derivative  with  respect  to 
b  is  found. 

dVar(y(6))  =  2frVar(x)-2Cov(7,x)  (2.6) 

db 


From  equation  (2.6)  the  minimum  point  is  found  by  setting  the  derivative  to  zero. 

Equation  (2.7)  is  the  candidate  for  b  that  minimizes  equation  (2.4)  thus  labeled  fi 

„  Cov(F.A-) 

P  Var(x) 

To  verify  this  is  a  minimum,  the  second  derivative  is  found. 


(2.7) 


^M  =  2Var(Y) 
db2  v  ' 
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Since  the  variance  is  always  positive,  then  equation  (2.7)  is  the  value  for  b  that  minimizes 
equation  (2.4).  Combining  equation  (2.4),  equation  (2.7),  and  using  some  simple  algebra 
yields  equation  (2.9). 

V  ar(y(/?))  =  (l  -  pZxy  )v  ar(y )  (2.9) 

Where  pXY  is  the  correlation  coefficient  between  X  and  Y.  Var(Y(//))  is  the  minimum 
variance.  The  controlled  observations  (2.10)  are  averaged  (2.1 1)  to  obtain  an  unbiased 
estimator  of  juY . 

yl(f>)=y,-p(x-flx\i= (2.io) 

(2.1D 

where  K  is  the  sample  size. 

Since  the  value  of  ft  is  unknown,  it  must  be  estimated.  An  estimate  of  ft  can  be 
found  by  substituting  the  sample  quantities  into  equation  (2.7).  This  solution  is  the  least 
squares  solution  for  [}.  The  least  squares  solution  is  also  the  maximum  likelihood 
solution  with  the  assumption  of  joint  nonnality  between  X  and  Y.  Equation  (2.12) 
estimates  J3. 

tb-rfo-x) 

XJ=i-r- - - -  (2.12) 

Z  (x.-xl 

i= 1 

The  point  estimator  of  /uY  is  estimated  by  equation  (2.13). 

(2-13) 

&  i= 1 
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Using  regression  theory  an  interval  estimate  for  //>  can  be  obtained.  By  making 
the  assumption  of  joint  nonnality  for  X  and  Y,  the  conditional  distribution  of  Y  given  X 
will  be  normal  by  equation  (2.14) 

Y  |  X  =  x  ~  N(//y  + /3(x- jUx),crl)  (2.14) 

where 

<js  =  <JY(l  — Pxy)  (2.15) 

and 

<Tj2  =  Var(r)  (2.16) 

Since  the  values  of  the  control  variable  X  and  it  mean  fix  are  known,  then  it  can  be  seen 
that  the  conditional  mean  of  Y  given  X has  two  terms.  The  terms  are  broken  into  the 
parameter  to  be  estimated  juY  and  a  correction  term.  To  get  the  juY  term,  the  corrections 
need  to  be  subtracted  out  as  in  equation  (2.10).  From  equation  (2. 14),  equation  (2.17)  can 
be  formed. 

Yi=iuY+p{Xi-iux)+£i,\<i<K  (2.17) 

where  si  are  the  residuals  and  are  of  the  form  in  equation  (2.18). 

si  ~  n(o,  cr2 )  (2.18) 

Since  the  values  of  juY  and  ft  are  unknown,  the  method  of  least  squares  can  be  applied  to 
solve  for  them.  //>  will  be  the  intercept  and  is  normally  distributed  as  in  equation  (2.19). 

~n(//7,ct2Sii)  (2.19) 

The  value  of  ,s'M  in  equation  (2.19)  is  the  upper  left  hand  entry  in  the  matrix  (pT D)  * 
where  D  is  of  the  form  in  equation  (2.20). 
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(2.20) 


D  = 


1 

1 

1 


^  \  ~  Mx 
X  2  ~  Mx 
^3  —  Mx 


1  X  K  fix 


To  generate  a  confidence  interval  about  juY 


a:  must  be  estimated.  Since  cr 


represents  the  variability  in  Y  given  X,  the  formula  for  the  residual  mean  square  error  is 
used. 


<j 


2  _  i=l 


K-2 


(2.21) 


where 


%[£)=  juY(p)+  PiX;- iux\\<i<K 


It  can  be  seen,  from  the  above  equations,  that 


°lsu 


/f Y 
1 


K-2 


t 


K-2 


(2.22) 


(2.23) 


has  a  Student-t  distribution  with  K-2  degrees  of  freedom.  Therefore,  the  confidence 
interval  for  juY  is  given  by 


My  ±t 


K- 2,  1 


ii 


(2.24) 


Now  in  simulations  there  are  possibly  more  than  one  control  for  a  response. 
Therefore,  equation  (2.17)  is  modified  to  be 
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(2.25) 


Y,  =  Mi  + 1>, (x„  -nXj )+  £„  1  <i<K 

j= 1 


where, 


S<^-1 


Therefore,  equation  (2.21)  is  refonned  into 


I 


2  _  '=1  y  V 


T- 


cr, 


;=i 


x\2 

I 


K-Q- 1 


(2-26) 


(2.27) 


r  V1 

and  5n  is  the  upper  left  hand  entry  in  the  matrix  \  D  D)  where  D  is  of  the  form 


'1 

X\  1  ~  MlX 

"^21  ~  M2X 

XQ1  MqX 

1 

X12  ~  Mix 

Xii~  Mix 

XQ1  ~  MqX 

D  = 

1 

X]3  —  Mix 

x 21  —  Mix 

to 

1 

7= 

1 

A 
•  1 

Xn~  Mix 

1 

•  1 

,  St 

(2.28) 


Then  the  100(l-«)%  confidence  interval  in  equation  (2.24)  becomes 


fiY±t 


K-Q- 1  1- 


i 


Vesn 


(2.29) 


An  important  factor  in  the  proper  use  of  the  CV  technique  is  the  selection  of 
controls.  In  selecting  potential  controls,  there  are  two  things  to  look  for.  The  first  is  an 
input  of  a  random  number  or  random  variate.  The  input  can  be  anything  from  a  draw 
from  a  random  distribution  to  a  conditional  switch.  The  second  item  of  importance  is  that 
the  expected  value  of  the  random  input  is  known.  There  are  many  ways  to  determine 
what  the  expected  value  is,  depending  on  the  random  input  parameters.  A  table  of 
expected  values  for  distributions  in  THUNDER  is  presented  in  Appendix  (B). 
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After  all  the  potential  candidates  for  controls  are  identified,  the  average  of  all  the 
realizations  of  each  potential  candidate  must  be  calculated  and  output  in  addition  to  the 
MOE  of  interest  for  each  replication.  Since  the  correlation  between  the  control 
candidates  and  the  MOE  is  unknown,  a  stepwise  regression  must  be  perfonned  to  identify 
which  control  candidates  are  significantly  correlated  to  the  selected  MOE. 

2.4.2.  Common  Random  Numbers. 

Common  Random  Numbers  (CRN),  also  known  as  correlated  sampling,  is  a 
relatively  intuitive  VRT.  The  rationale  behind  CRN  is  to  induce  a  positive  correlation  in 
the  random  inputs  between  two  or  more  configurations  of  a  study.  The  positive 
correlation  will  diminish  the  noise  generated  by  the  difference  in  random  inputs  and  in 
turn  reduce  the  variance  in  the  difference  between  the  outputs.  To  create  the  positive 
correlation,  the  two  configurations  should  use  the  same  random  variable  inputs.  The 
following  theory  behind  CRN  was  adapted  from  Law  and  Kelton.  (20:582-598) 

To  understand  CRNs,  consider  two  alternative  configurations  of  a  single 
simulation,  where  Xxj  and  X2j  are  values  from  the  first  and  second  configuration  on  the 

/h  independent  replication.  Also  consider,  the  estimate  in  question  is  equation  (2.30). 

Z  =  lh =e[aJ-e[x2J  (2.30) 

Let  there  be  n  replications  of  each  system  and  let 

Zj  =  Xxj  -X2J,j  =  1,2,..., n  (2.31) 

then 

e[zJ  =<r  (2.32) 

Therefore,  equation  (2.33)  is  an  unbiased  estimator  of  £ 
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Since  the  Z,  ’s  are  independent,  identically  distributed  (IID)  random  variables, 

V:,r(z(„))  Var(Z-)  Varb,J>V^feJ-2C°v(-y.J-y3J)  (234) 

n  n 

If  Xly.  and  X2j  are  independent,  then 

Cov(xi;,Xy)=0  (2.35) 

If  then  Xxj  and  X2j  are  not  independent  and  are  positively  correlated 

Cov(xiy.,X2y.)>0  (2.36) 

then  Var(z(«))  will  be  reduced. 

The  key  to  the  implementation  of  CRN  is  synchronization.  The  random  number 
streams  should  match  up  between  the  two  configurations  as  much  as  possible.  This 
means,  that  a  random  number  drawn  should  be  used  for  the  same  purpose  in  both 
configurations.  Ideally,  all  the  random  variate  draws  should  be  applied  in  exactly  the 
same  way  for  both  configurations.  To  synchronize  the  simulations,  some  initial  coding  is 
involved.  First,  all  of  the  points  in  the  simulation  where  a  random  number  or  variate  is 
drawn  must  be  identified.  Then,  each  point  is  then  assigned  its  own  random  number 
stream. 

A  problem  that  may  arise,  when  coding  for  CRN,  is  the  sufficiency  of  the  RNG. 

A  primary  concern  should  be  how  to  keep  track  of  all  the  different  random  number 
streams.  Some  large  simulations  like  THUNDER  could  require  hundreds  of  streams. 
Ideally,  each  of  the  streams  should  be  independent  and  not  overlap.  This  could  be 


difficult,  because  these  simulations  could  have  millions  or  even  billions  of  random 
number  draws.  Therefore,  a  large  RNG  with  built  in  functions  that  keep  track  of  random 
number  streams  is  beneficial. 

2.4.3.  Antithetic  Variates. 

Antithetic  variate  (AV)  VRT  is  similar  to  CRN.  As  in  CRN,  correlation  is 
induced  between  separate  runs,  but  this  time  the  object  is  to  induce  a  negative  correlation. 
The  main  difference  is  that  instead  of  having  two  configurations  of  one  system,  there 
only  needs  to  be  one  system.  The  following  is  an  excerpt  from  Law  and  Kelton(20:598- 
604). 

The  basic  idea  is  to  make  pairs  of  runs  of  a  model  where  if  the  first  run  observes  a 
“small”  observation,  then  the  second  run  observes  a  “large”  observation.  Therefore,  the 
two  observations  are  negatively  correlated.  To  obtain  the  negative  correlation, 
complementary  numbers  are  used.  For  example,  if  Uk  is  a  particular  random  number 

used  for  a  particular  purpose  in  the  first  run,  then  1  -  U k  is  used  for  the  same  particular 
purpose  on  the  second  run.  The  use  of  l-Uk  is  valid,  since  U  ~  U(0,1)  then 
1  -U  ~  U(0,1)  as  well.  Note  that  like  CRN,  synchronization  is  extremely  important  to 
the  successful  use  of  AV. 

To  understand  the  basis  for  AV,  suppose  there  are  n  pairs  of  runs  in  a  simulation 
resulting  in  observations  (a®,  Aj(2)),  ...,  (a,^, A‘2)),  where  Aj.1’  is  from  the  normal  run 

of  the  jth  pair,  and  A'2'  is  from  the  antithetic  run  of  the  jth  pair.  Both  A'1’  and  A'2'  are 
legitimate  observations  of  the  simulation,  therefore 
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eM=e[*;2)]  =h 


(2.37) 


Since  each  run  is  independent,  each  pair  of  runs  is  independent.  Now  let 

fc’+jrf) 

J~  2 

and  let 

X(n)='-±X1 
n  J= i 

be  the  unbiased  point  estimator  of 

//  =  E[jr<'>]=E[jrJ=E  [x(n)\ 

Since  the  X .  ‘s  are  IID, 

Var[x(„)l  =  Var[Z>]  =  Var[z;'l]+Varh;2>]+2Cov[z;",Zf' ] 
n  4  n 

If  a  negative  correlation  could  be  induced  between  X1'1  and  X(r] ,  then 


(2.38) 


(2.39) 


(2.40) 


(2.41) 


Cov[xj.1),Xj2)]  <0  (2.42) 

which  reduces  Var[x(n)]  . 

2.4.4.  Control  Variates  and  Antithetic  Variates. 

Kwon  and  Tew  (15)  compared  three  methods  of  combining  Control  Variates  and 
Antithetic  Variates.  The  first  method  consists  of  using  CV  on  a  simulation  while  using 
AV  only  on  the  random  components  not  used  in  the  CV  technique.  The  second  method 
consists  of  using  CV  on  a  simulation  while  using  AV  only  on  the  random  components 
used  in  the  CV  technique.  The  third  method  consists  of  using  CV  on  a  simulation  while 
using  AV  on  all  of  the  random  components  in  the  simulation.  Kwon  and  Tew  concluded 
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that  under  general  assumptions,  the  combined  methods  yield  superior  perfonnance.  They 
also  found  that  the  third  method  outperformed  the  other  two.  Method  three  is  the  method 
applied  to  THUNDER  for  this  research. 

The  basic  application  behind  method  three  of  Kwon  and  Tew  is  to  obtain  the 
average  response  and  the  mean  of  the  control  variates  across  h  pairs  of  replicates.  The 
theory  follows  from  Kwon  and  Tew  (15).  Four  properties  are  derived. 

1.  Since  the  random  inputs  into  the  simulation  are  essentially  independent,  there  is 
Homogeneity  of  response  variances  across  replicates, 

Var [y, ]  =  <r2y  for  i  =  1,  2,  . . .,  2 h  (2.43) 

2.  Antithetic  effect:  there  is  homogeneity  of  response  correlations  across  replicate  pairs 
and  independence  of  responses  between  replicate  pairs. 

Cov[y,,y,]  = 

Note:  the  correlation  across  replicate  pairs  is  a  negative  correlation.  Thus,  p  is 
positive. 

3.  Control  Variate  and  Antithetic  effect:  there  is  homogeneity  of  response-control 
variates  covariance  across  replicates,  homogeneity  of  response-control  variates 
covariance  across  replicate  pairs,  and  independence  of  response-control  variates  on 
different  replicate  pairs. 

<jyc  for  i  =  k  =  1,2,. ..,2  h 

Cov[y,-,c;t]  =  <  cr[')  for  i  =  k  +  1  or  k  =  / +  l(min  {/,&}  =  l,3,...,2/z -l)  ►  (2.45) 

0(sxl)  otherwise 


-  pa;  for  i  =  k  +  \(k  =  1,3,. ..,2 h  - 1) 


otherwise 


(2.44) 
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4.  Antithetic  effect  also  gives  us  homogeneity  of  control  variates  covariances  across 
replicates,  homogeneity  of  control  variates  covariances  across  replicate  pairs,  and 
independence  of  control  variates  on  different  pairs  of  replicates. 


Cov[c,  ,ck\  = 


Sc 

z‘ 


(5x5) 


for  /  =  k  =  1,2,... ,2  h 

for  /  =  k  +  \(k  =  1,3,. ..,2 h  -  l)  > 

otherwise 


(2.46) 


The  following  equations  can  be  derived  from  the  four  properties  above. 

Varb/]=i(Var[y2,,]+  Var[>’2/]+  2Cov\y2j  ,,y2/ ])  =  |(l - p)a)  (2.47) 

Cov[c .  ]  =  ^(Cov[c2/_l]+  Cov[c2 .  ]+  2Cov[c2jH,c2, ])  =  |(ZC  +  Z?)  (2.48) 


and 


]=4Cov[t2.h  + 


yij’c2j-i 


(2.49) 


for  j  =  1,2,  . . . ,  h.  Under  the  joint  normality  assumption  and  properties  from  equations 
(2.43),  (2.44),  (2.45),  and  (2.46),  the  joint  distribution  of  y}  and  cy  is  given  as 


y  i 


N 


s+l 


f  r 


V- 


0(Sxl). 


(l  -P]°l  i^yc+^ycj 

(1)  z.+zl1’ 


ayc  +  ayc 


fory  =  1,2,  ...,  h 


J  J 


(2.50) 


Since  c  ( j  =  1,2,  ...,  h)  are  independent,  the  unconditional  variance  for  juv  is 


Var[/}J: 


2  h 


h-2 
h-s-  2 


(2.51) 


where 


^  =  (l  -  pWy  ~  (&yc  +  °yc  )T  +  ll1  ’ )  '  (<7yc  +  ^  ) 


(2.52) 
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Therefore,  a  reduction  in  variance  of  ju  will  occur  if  the  effects  due  to  AV  and 

CV  together  compensate  for  the  loss  factor  given  as  ((h-2)/(h-s-2)).  Where  s  is  the 
number  of  random  number  streams  used  to  derive  the  control  variate  random  numbers. 

2.4.5.  Control  Variates  and  Common  Random  Numbers. 

Yang  and  Nelson  (44)  developed  a  method  that  combines  CVs  and  CRNs  in 
applications  of  multiple  comparisons.  Multiple  comparisons  compare  the  expected 
values  of  univariate  performance  measures  common  to  two  or  more  systems  by 
constructing  simultaneous  interval  estimates  of  the  differences  in  expected  performance. 
Yang  and  Nelson  address  three  types  of  multiple  comparison  procedures.  They  are 
Tukey’s  all-pairwise-multiple  comparison  (MCA)  procedure,  Hsu’s  multiple 
comparisons- with-the -best  (MCB)  procedure,  and  Dunnett’s  multiple-comparisons-with- 
a-control  (MCC)  procedure.  MCA  is  used  to  determine  the  difference  between  the 
performance  of  each  system  design  and  every  other  system  design.  MCB  is  used  to 
identify  the  system  with  the  largest  mean  performance.  MCC  is  used  to  detennine  the 
difference  between  the  mean  performance  of  a  selected  system  design  and  every  other 
system  design.  Their  goal  was  to  refine  the  three  procedures,  using  the  variance 
reduction  techniques  of  CV  and  CRN,  to  increase  sensitivity  to  differences  in  system 
performance. 

The  idea  behind  Yang  and  Nelson’s  approach  is  to  use  the  following  control 
variate  estimators  to  estimate  the  mean  perfonnance  for  a  system  when  applying  multiple 
comparison  procedures. 

4  =Yi-/3'i(X-n)  for  i  =  1,2,  ...,r  (2.53) 
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where  r  is  the  total  number  of  systems,  T  is  the  mean  performance  measure,  /?,  is  the 
control  variate  estimates  generated  though  least  squares  estimates,  X  is  a  vector  of 
independent  random  variables  with  known  mean  vector  p.  An  assumption  required  to  do 
this  is  that  the  random  variables  X.j,  the  control  vectors,  are  available  and  identical  across 
all  systems.  Therefore  the  control  vecotors  must  be  available  in  each  system  design  so 
they  can  be  forced  to  take  identical  values  through  the  application  of  CRN.  The  other 
random  inputs  into  the  system  need  to  be  independent  to  develop  the  control  variate 
estimation. 

Yang  and  Nelson  modify  the  interval  estimates  for  all  three  multiple  comparison 
procedures  to  use  the  control  variate  estimation  which  relies  on  the  fact  that  the  control 
vectors  are  identical  across  all  systems.  They  then  develop  the  following  theorem 


I  X] 

E[H] 


-^Wl -R2  <1 


(2.54) 


where  E\HCV  |  X] 


is  the  expected  width  of  the  stadard  control  variate  interval  and 


e[h]  is  the  expected  width  of  the  multiple  comparison  interval.  The  equation 
calculates  how  the  difference  intervals  developed  by  the  control  variates  are  smaller  than 
the  normally  generated  difference  intervals. 

2.4.6.  Other  Techniques. 

There  are  many  other  VRTs  than  CV,  CRN,  and  AV.  In  December  of  1983, 
Lieutenant  Colonel  Mohamed  Refat  Elhefny  (6)  produced  a  thesis  comparing  many 
variance  reduction  techniques.  He  performed  an  extensive  literature  review  to  identify 
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and  learn  about  the  different  VRTs.  The  following  is  a  list  of  all  the  VRTs  covered  in 
LtCol  Elhefny’s  research. 

•  Importance  Sampling  •  Control  Variates 

•  Russian  Roulette  and  •  Antithetic  Variates 

Splitting  •  Regression 

•  Systematic  Sampling  •  Sequential  Sampling 

•  Stratified  Sampling  •  Orthogonal  Functions 

•  Expected  Value  •  Adjoint  Method 

•  Statistical  Estimation  •  Transformation 

•  Correlated  Sampling  •  Conditional  Monte  Carlo 

•  History  Reanalysis 

Within  his  research  he  created  a  table  that  outlines  the  characteristics  of  many  VRTs. 

The  table  has  been  reproduced  in  Appendix  (H).  The  table  includes  a  description,  criteria 
for  application,  advantages,  disadvantages,  and  typical  applications  for  each  technique. 
LtC  Elhefny  concluded,  “there  is  no  single  technique  which  is  the  most  suitable 
technique  for  every  simulation  problem.”(6:VI-l)  He  also  thought  that  research  should 
be  perfonned  to  identify  VRTs  that  may  be  affective  for  specific  simulations.  This  could 
aid  the  analysis  by  identifying  VRTs  that  may  work  with  a  simulation. 

2.4.7.  Previous  Variance  Reduction  Studies. 

There  have  been  many  studies  in  variance  reduction  techniques.  Curtis  Smith 
from  the  Naval  Postgraduate  School  (NPS)  set  out  to  identify  VRTs  that  are  applicable  to 
a  class  of  stochastic  combat  simulations  and  then  exhibit  their  performance  in  the 
applications.  He  applied  the  Antithetic  Variates  and  Stratified  Sampling  techniques  to  a 
simple  stochastic  combat  simulation.  He  proposed  that  both  of  these  techniques  are 
capable  of  variance  reduction  in  stochastic  combat  models.  Since  the  work  was  on  a 
simple  stochastic  combat  simulation,  it  was  recommended  that  further  studies  on  larger, 
real  world  simulations  be  perfonned.  (3 1) 
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2.5.  Random  Number  Generator 


The  random  number  generator  (RNG)  is  an  integral  part  of  a  stochastic 
simulation.  Therefore,  it  should  be  of  great  interest  to  the  analyst  as  to  what  RNG  is  used 
for  a  simulation  and  how  well  it  performs.  Currently,  most  programming  software  use 
small  linear  congruential  generators  (LCGs)  as  the  random  number  generators.  It  is 
recognized  by  simulation  experts  that  small  LCGs,  with  moduli  around  231,  should  not  be 
used  for  most  purposes.  In  fact,  the  period  on  most  any  LCG  can  be  exhausted  in  a 
matter  of  minutes  on  a  standard  PC  and  LCGs  are  known  to  have  poor  point  structure, 
both  of  which  can  bias  simulation  results.  L’Ecuyer,  proposed  to  replace  the  more 
common  LCGs  with  a  more  robust  RNG  method  called  a  combined  multiple  recursive 
generator  (CMRG).  L’Ecuyer,  Simard,  Chen,  and  Kelton  have  produced  a  CMRG  RNG 
package,  which  is  readily  available  on  the  Internet  and  is  reviewed  below.  (18) 

A  parameter  set  for  a  CMRG,  from  a  previous  paper  by  L’Ecuyer,  was  selected  as 
the  backbone  of  the  proposed  RNG  package.  The  parameter  set  is  called  MRG32k3a  and 
was  one  of  many  sets  that  were  discovered  after  a  years  worth  of  computer  processing 
time  (16).  MRG32k3a  has  a  long  period  of  approximately  2 191  or  3.1xl057.  RNGs  with 
longer  periods  have  been  proposed,  but  MRG32k3a  is  more  efficient  with  today’s 
computers  and  the  period  is  long  enough  not  to  be  depleted  by  a  years  worth  of 
computing.  To  remain  efficient,  a  parameter  set  was  chosen  that  had  the  ability  to  use 
floating-point  arithmetic.  Floating-point  arithmetic  is  much  faster  to  perform  than  regular 
arithmetic.  MRG32k3a  is  also  very  sound  statistically.  The  random  number  stream  from 
MRG32k3a  is  so  uniform;  they  pass  the  spectral  test  in  at  least  45  dimensions.  This 
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means  that  when  generating  sets  of  t  numbers,  they  would  cover  a  /-dimensional  unit 
hypercube  very  uniformly. 

The  implementation  of  MRG32k3a  is  relatively  simple.  It  maintains  only  2 
components,  each  of  order  3.  The  components  are  stored  in  vectors.  At  step  n,  the 


vectors  would  be  as  follows. 

S\ji  ~  (-T,«  ’  Xl,n+\  ’  X\,n+2  )  (2.55) 

S2,n  =  (X2,n  ’  X2,n+1  >  X2,n+2  )  (2.56) 

The  vectors  evolve  according  to  the  equation  (2.57)  and  (2.58). 

xln  =  (l403580xx1(I_2  -810728xx1)I_3)mod4294967087  (2.57) 

x2n  =  (527612xxln_j -1370589xxj  H_3)mod  4294944443  (2.58) 

Then  the  output  u„  is  defined  by  the  following  relationship. 

f  zj  4294967088  ifz„  >0 
U”  “[4294967087/4294967088  ifz„=0  (2'59) 

where 

zn  =  [x\,n  -x2  „)mod 4294967087  (2.60) 


To  create  multiple  streams,  the  full  period  needed  to  be  broken  into  smaller 
sections  that  are  equal  in  length.  The  goal  was  to  find  a  size  for  the  streams  and 
substreams  that  are  relatively  long  and  have  good  statistical  properties.  In  essence  each 
stream  and  sub  stream  should  resemble  a  stand  alone  random  number  generator.  To  do 
this,  the  statistical  structure  of  many  different  streams  and  substreams  all  ranging  in 
length,  between  251  and  2150  were  examined.  The  combination  of  each  stream  consisting 
of  2  numbers,  with  2  substreams  consisting  of  2  ’  numbers  was  chosen.  Since  the 
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streams  and  substreams  are  of  equal  length,  four  3x3  matrices  were  developed  so  that 
the  software  could  advance  the  state  of  the  RNG  by  either  2 127  or  276  numbers  with  a 
single  matrix  multiplication.  This  allows  the  software  to  immediately  jump  to  the  next 
stream  or  substream. 

To  maintain  the  proper  synchronization  within  the  RNG,  the  computer  keeps  track 
of  the  beginning  of  the  current  stream,  the  beginning  of  the  current  substream,  the 
beginning  of  the  next  substream,  and  the  current  state  of  the  stream  for  each  stream.  This 
allows  the  user  to  reset  the  stream,  reset  the  substream,  jump  to  the  next  stream  or  jump 
to  the  next  substream  without  loosing  continuity. 

The  speed  of  the  MRG32k3a  generator  was  compared  with  a  commonly  used 
LCG,  lcgrand  from  Law  and  Kelton  (16).  A  total  of  109  random  numbers  were  generated 
from  both  RNGs  and  were  timed.  The  RNGs  were  run  on  a  600MHz  PHI  with  128MB  of 
RAM,  running  Windows  98.  MRG32k3a  completed  the  run  in  12.9  minutes  while 
lcgrand  completed  the  run  in  14.8  minutes.  Even  though  the  MRG32k3a  has  more  than 
twice  as  many  calculations  as  a  LCG,  it  still  runs  faster.  Primarily,  this  is  because  the 
code  for  MRG32k3a  implements  floating-point  arithmetic.  To  implement  floating-point 
arithmetic,  no  variable  or  intermediate  variable  can  exceed  the  precision  of  the  computer 
it  is  running  on,  which  is  generally  32  bits.  To  make  sure  the  calculations  do  not  exceed 
the  precision  of  the  computer,  the  backbone  equations  have  been  slightly  modified  to 
store  the  intermediate  calculation  variables  in  smaller  variables. 

The  RNG  package  includes  many  extra  utilities  to  aid  the  use.  First,  the  seed 
values  for  the  whole  package  or  individual  stream  can  be  changed.  Second,  each  stream 
can  be  set  to  produce  Antithetic  variables  for  variance  reduction.  Third,  the  stream  can 
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be  reset  to  the  start  of  the  stream  or  substream.  Finally,  the  precision  of  the  uniform 
random  number  can  be  increased  from  32-bit  precision  to  53-bit  precision.  A  list  of  C 
routines  is  located  in  Appendix  (G). 

2.6.  Major  Shared  Resource  Center  and  Divide  and  Conquer 

The  ASC  MSRC  is  a  computational  science  facility.  It  supports  various  DoD 
research  including  development,  test,  and  evaluation.  As  part  of  the  DoD's  High 
Performance  Computing  (HPC)  Modernization  Program,  the  ASC  MSRC  uses  high 
performance  computing  research  and  visualization.  To  provide  an  interface  between  the 
analyst  and  HPC  computing  resources,  the  Divide  and  Conquer  (D&C)  system  was 
developed.  D&C  has  a  web  based  Graphical  User  Interface  (GUI)  that  allows  an  analyst 
to  specify  parameters  for  a  study.  D&C  will  take  the  study  parameters  and  automatically 
set  up  all  requested  replications  and  directories.  After  setting  up  the  replications,  D&C 
queues  the  replication  requests  in  the  specified  HPC  resource. 

2.7.  Matlab  and  Regression 

Matrix  Laboratory  (MATLAB®)  is  a  high-performance  language  for  technical 
computing.  It  was  developed  to  provide  easy  access  to  matrix  software,  embedding  state 
of  the  art  software  for  matrix  computation.  The  basic  data  element  is  an  array  that  does 
not  require  dimensioning,  giving  the  ability  to  solve  many  technical  computing  problems, 
specifically  those  with  matrix  and  vector  formulations,  without  the  need  to  write  a 
program  in  a  scalar,  noninteractive  language  such  as  C,  Visual  Basic,  or  Fortran. 

MATLAB  features  a  family  of  add-on  application-specific  solutions  called 
toolboxes.  Toolboxes  are  collections  of  MATLAB  functions  that  extend  the  MATLAB 
environment  to  solve  particular  classes  of  problems.  One  of  the  MATLAB  toolboxes  is 
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the  statistics  toolbox  (24).  The  statistics  toolbox  is  a  collection  of  statistical  tools  built  on 
the  MATLAB  computing  environment.  The  toolbox  supports  a  wide  range  of  common 
statistical  tasks,  from  random  number  generation,  to  curve  fitting,  to  design  of 
experiments,  statistical  process  control,  and  regression  (25). 

A  useful  function  from  the  statistical  toolbox  is  stepwisefit.  The  following 
information  about  stepwisefit  was  taken  from  the  MATLAB  statistics  documentation. 
Stepwisefit  uses  stepwise  regression  to  model  the  response  variable  y  as  a  function  of  the 
predictor  variables  represented  by  the  columns  of  the  matrix  X.  A  standard  call  to  the 
stepwisefit  function  is  [b ,  se,  pval,  inmodel,  stats,  nextstep, 
history]  =  stepwisefit (X,  y,  ' Paraml ' ,  valuel,  ' Param2 ' , 
value2  ,  .  .  . ) .  Where X,y,  ’Paraml’,  valuel,  ’Param2’,  value2,  etc.  are  function 
parameters  and  b,  se,pval,  inmodel,  stats,  nextstep,  and  history  are  function  results. 

Table  2.1  contains  a  description  of  the  stepwisefit  parameters,  b  is  a  vector  of  estimated 
coefficient  values  for  all  columns  of  X.  The  b  value  for  a  column  not  included  in  the  final 
model  is  the  coefficient  that  you  would  obtain  by  adding  that  column  to  the  model,  se  is 
a  vector  of  standard  errors  for  b.  pval  is  a  vector  of  p-values  for  testing  whether  b  is  0. 
inmodel  is  a  logical  vector,  whose  length  equals  the  number  of  columns  inX,  specifying 
which  predictors  are  in  the  final  model.  A  1  in  position  j  indicates  that  the  /th  predictor  is 
in  the  final  model;  a  0  indicates  that  the  corresponding  predictor  in  not  in  the  final  model, 
stats  is  a  structure  containing  additional  statistics,  nextstep  is  the  recommended  next 
step,  either  the  index  of  the  next  predictor  to  move  in  or  out,  or  0  if  no  further  steps  are 
recommended,  history  is  a  structure  containing  information  about  the  history  of  steps 
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taken.  The  following  table  specifies  one  or  more  of  the  name/value  pairs  used  as 
parameters  (23). 


Table  2.1:  Parameter  Specifications  for  Stepwisefit  (23) 


Parameter  Name 

Parameter  Value 

'inmodel' 

Logical  vector  specifying  the  predictors  to  include  in  the  initial  fit.  The 
default  is  a  vector  of  all  zeros,  specifying  no  predictors. 

'penter' 

Maximum  p-value  for  a  predictor  to  be  added.  The  default  is  0.05. 

Minimum  p-value  for  a  predictor  to  be  removed.  The  default  is  0. 1 0. 

'display' 

'on'  displays  information  about  each  step. 

'off  omits  the  infonnation. 

'maxiter' 

Maximum  number  of  steps  to  take  (default  is  no  maximum) 

'keep' 

Logical  vector  specifying  the  predictors  to  keep  in  their  initial  state. 

The  default  is  a  vector  of  all  zeros,  specifying  no  predictors. 

'scale' 

'on'  scales  each  column  of  X  by  its  standard  deviation  before  fitting. 

'off  does  not  scale  (the  default). 

2.8.  Analysis 

The  following  subsections  contain  calculations  used  in  the  analysis  presented  in 
chapter  4.  The  first  subsection  describes  how  to  generate  a  halfwidth  and  a  confidence 
interval.  The  second  subsection  gives  the  calculations  for  determining  the  number  of 
replications  needed  to  achieve  a  specified  halfwidth.  The  third  subsection  outlines  the 
calculations  used  to  detennine  a  paired-t  CL  The  last  subsection  describes  the  Bonferroni 
inequality  and  how  to  calculate  it. 

2.8.1.  Confidence  Intervals. 

Confidence  intervals  (CIs)  are  used  to  determine  the  1 00(  I  -a)%  bound  on  an 
estimated  mean.  The  Cl  is  based  heavily  on  the  variance  of  the  mean  estimate.  The 
standard  equation  for  a  Cl  is  below. 

Cl  =  /jx±S(n,a)  (2.61) 
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where 


Mx  — 


n  ,=1 


(2.62) 


and 

M'ua)=t,_  (2.63) 

V  22 

and  n  is  the  number  of  sample  points,  S2(n)  is  the  variance  of  the  sample  points,  and 
tn-i,\-a/2  is  the  upper  1  -  a  /  2  critical  point  for  the  /-distribution  with  n  -  1  degrees  of 


freedom.  Equation  (2.63)  is  called  the  halfwidth  (20:5 1 1-513). 

2.8.2.  Determining  the  Number  of  Replications. 

To  calculate  the  estimated  number  of  replications  needed  to  achieve  a  specified 
halfwidth  fi  the  following  equation  would  be  used. 


z*  (/?)  =  mini  i  >  n  :  t 


s2(n ) 


i-\,\—a!  2 ' 


(2.64) 


To  use  the  above  equation,  it  is  assumed  that  5,2(n)  is  constant  for  all  n  (20:512). 

2.8.3.  Paired-t  Confidence  Interval. 

The  paired-t  Cl  is  a  method  of  comparing  two  systems  on  the  basis  of  a 
performance  measure.  The  paired-t  Cl  calculates  a  Cl  about  the  difference  between  the 
performance  measures.  If  the  Cl  contains  zero  then  the  performance  measures  from  both 
systems  can  be  assumed  equivalent. 

Lets  say  there  are  n\  replications  of  a  perfonnance  measure  of  interest  one  X\j 
where  j  ranges  from  1  to  221  and  there  are  222  replications  of  a  performance  measure  of 
interest  one  X2 k  where  k  ranges  from  1  to  222.  If  ni  =  222  =  n  then  we  can  define 
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Zi  =X1J-X1J,j  =  l,2,...,n 


(2.65) 


Since  the  Xy  are  taken  from  replications,  the  Z/s  are  IID  random  variables  and  the 
estimator  for  expected  value  of  Z  is 


Z(n) 


and  the  estimator  for  the  variance  is 


(2.66) 


Var[z(n)] 


M_ _ 

n(n- 1) 


Now,  to  form  the  100(1  -  a)%  Cl 

ci[z(«)]=  z(»)±tM2X4 W)\ 

Where  the  halfwidth  is  defined  as 


S(n,a)  =  tn_tl_alJVar[z(n)\ 


(2.67) 


(2.68) 


(2.69) 


2.8.4.  Bonferroni  Inequality. 

In  most  simulations,  an  analyst  is  interested  in  several  simultaneous  performance 
measures.  Suppose  that  4  is  a  100(1  -  a)%  Cl  for  the  measure  of  performance  jus  where  s 
=  1,2,  k.  The  probability  that  all  k  CIs  simultaneously  contain  their  respective  true 
measures  is 


PUS  e  =  1,2,  •  •  • ,  k)  >  1  -  £  a,  (2.70) 

s= 1 

Whether  or  not  the  //s  are  independent.  This  is  known  as  the  Bonferroni  inequality. 
Therefore,  to  maintain  a  simulation  level  of  a,  choose  a/s  such  that 
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(2.71) 


k 

a  =  'J'a^s  =  1,2,..., A: 

s= 1 

Chapter  3  uses  the  basic  understanding  of  the  different  tools  and  techniques 
developed  in  this  chapter.  It  describes  how  the  techniques  are  applied  and  which  tools 
are  used  in  the  application. 
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3.  Methodology 


This  chapter  begins  with  the  research  objective.  It  then  transitions  into  choice  of 
THUNDER  model  databases  and  MOEs,  describing  how  the  MOEs  are  collected  for 
analysis.  Next,  there  is  a  description  of  how  the  VRTs  are  performed.  Finally,  it  finishes 
with  a  discussion  of  how  the  analysis  replications  are  run  at  the  Major  Shared  Resource 
Center  (MSRC)  through  the  Divide  and  Conquer  (DAC)  web  interface. 

3.1.  Research  Objective 

The  objective  of  this  research  is  to  study  the  use  of  three  variance  reduction 
techniques  on  THUNDER.  The  three  techniques  are  Control  Variates,  Common  Random 
Numbers  and  Antithetic  Variates.  There  are  three  basic  steps  to  obtain  the  research 
objective.  First,  identify  a  set  of  MOEs  that  are  commonly  used  by  THUNDER  users. 
This  will  guarantee  that  the  results  will  be  of  some  interest  to  the  users.  Second,  modify 
the  THUNDER  source  code  to  implement  the  three  techniques.  This  consists  of 
identifying  areas  where  random  variates  are  drawn,  creating  an  output  file  of  the  mean 
value  for  the  random  inputs  and  implementing  a  new  random  number  generator.  Third, 
analyze  the  results  from  the  variance  reduction  techniques  to  determine  which,  if  any,  are 
effective  and  if  effective  how  efficient  the  technique  is  when  applied  to  the  model. 

3.2.  Model  Selection  for  THUNDER  Analysis 

There  are  three  notional  model  databases  that  are  used  as  examples  and  come  with 
THUNDER.  They  are  called  Databig,  Datasmall,  and  ME.  Most  THUNDER  model 
databases  are  classified  due  to  the  amount  and  the  nature  of  the  data  in  the  model. 
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Therefore,  one  of  the  notional  models  was  selected  for  the  THUNDER  analysis  so  that 
this  research  would  remain  unclassified.  There  is  little  documentation  about  any  of  the 
three  example  models,  except  that  one  of  the  three  data  sets  is  a  European  scenario,  while 
the  other  two  data  sets  are  Middle  Eastern  scenarios.  Since  the  THUNDER  introductory 
course  uses  the  ME,  a  Middle  Eastern  scenario,  database  (7),  it  was  selected  to  be  the 
representative  scenario  for  this  research.  It  is  a  5 -day  scenario  consisting  of  blue  forces 
from  Saudi  Arabia  and  red  forces  from  Iraq. 

Running  the  standard  package  ME  scenario  is  applicable  for  both  CV  and  AV, 
since  they  both  are  single-model  VRTs.  On  the  other  hand,  the  standard  application  of 
CRN  requires  at  least  two  configurations  or  models  to  be  compared.  Therefore,  a 
modification  to  the  original  ME  configuration  is  needed.  Because  little  is  known  about 
the  scenario,  a  file  that  should  have  a  significant  effect  on  the  outcome  of  the  MOEs  was 
chosen.  That  file  selected  was  squadron.dat.  It  contains  information  pertaining  to 
squadrons  in  the  scenario.  To  modify  it  significantly,  204  F-l  1 1  aircrafts  were  taken 
from  the  Blue  forces.  This  eliminated  all  of  the  F-l  1 1  from  the  scenario.  No 
modifications  were  made  to  any  other  ME  input  files.  A  section  of  Appendix  C  of  the 
THUNDER  Analyst  Manual  (33),  pertaining  to  the  squadron.dat  file  is  located  in 
Appendix  (I). 

3.3.  Measure  of  Effectiveness  Selection 

The  MOEs  for  the  ME  scenario  were  selected  after  discussion  with  expert 
THUNDER  users  (32).  Three  types  of  MOEs  that  seem  to  interest  the  interviewed  users 
are  forward  line  of  troops  (FLOT)  movement,  sortie  generation  and  aircraft  attrition. 
These  MOEs  can  be  extracted  from  standard  THUNDER  reports.  The  two  types  of 
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reports  needed  are  Ground  Combat  Cycle  reports  (CC)  and  the  Air  War  reports  (AW). 
The  output  data  for  the  FLOT  movement  is  located  in  the  CC-4  (FLOT  Movement) 
reports,  the  data  for  sorties  generated  is  located  in  the  AW-5  (Sortie  Generation)  reports 
and  the  data  for  aircraft  attrition  is  located  in  the  AW-3  (Aircraft  Loss  Summary)  reports. 
All  three  reports  are  contained  within  a  single  output  file  named  reports. std.  A  one-day 
sample  of  these  reports  is  located  in  Appendix  (A).  A  set  of  2 1  specific  MOEs  was 
selected  from  the  three  reports.  The  following  table  identifies  all  MOEs,  aggregation 
level,  force,  and  the  reports  they  are  calculated  from. 

Table  3.1:  Measure  of  Effectiveness  Table 


MOE 

AGGREGATION 

FORCE 

REPORT 

A-10  Losses  perSortie  (LPS) 

System  Level 

BLUE 

AW-3 

A-6  Losses  perSortie  (LPS) 

System  Level 

BLUE 

AW-3 

BLUE  Flown  per  Planned  Sortie  (FPS) 

Force  Level 

BLUE 

AW-5 

BLUE  Losses  perSortie  (LPS) 

Force  Level 

BLUE 

AW-3 

BLUE  Square  Miles  Gained  (SMG) 

Force  Level 

BLUE 

CC-4 

DHAHRAN  Flown  per  Planned  Sortie  (FPS) 

Base  Level 

BLUE 

AW-5 

F-111  Losses  perSortie  (LPS) 

System  Level 

BLUE 

AW-3 

F-15  Losses  perSortie  (LPS) 

System  Level 

BLUE 

AW-3 

FA-18  Flown  per  Planned  Sortie  total  (FPS  total) 

Squadron  Level 

BLUE 

AW-5 

FA-18  Flown  per  Planned  Sortie  DCA  (FPS  DCA) 

Squadron  Level 

BLUE 

AW-5 

MIG-23  Losses  perSortie  (LPS) 

System  Level 

RED 

AW-3 

MIG-29  Losses  per  Sortie  (LPS) 

System  Level 

RED 

AW-3 

MIRAGE  Flown  per  Planned  Sortie  total  (FPS  total) 

Squadron  Level 

RED 

AW-5 

MIRAGE  Flown  per  Planned  Sortie  INT  (FPS  INT) 

Squadron  Level 

RED 

AW-5 

MIRAGE  Losses  perSortie  (LPS) 

System  Level 

RED 

AW-3 

MUDAYSIS  Flown  per  Planned  Sortie  (FPS) 

Base  Level 

RED 

AW-5 

RED  Flown  per  Planned  Sortie  (FPS) 

Force  Level 

RED 

AW-5 

RED  Losses  perSortie  (LPS) 

Force  Level 

RED 

AW-3 

RIYADH  Flown  per  Planned  Sortie  (FPS) 

Base  Level 

BLUE 

AW-5 

SHAIBAH  Flown  per  Planned  Sortie  (FPS) 

Base  Level 

RED 

AW-5 

For  this  research,  all  of  the  selected  MOEs  were  calculated.  To  calculate  the 
Losses  per  Sortie  (LPS)  MOEs  for  the  specific  system,  squadron,  base  or  force,  the  total 
number  of  losses  were  divided  by  the  total  number  of  sorties.  To  calculate  the  Flown  per 
Planned  Sortie  (FPS)  MOEs  for  the  specific  system,  squadron,  base  or  force,  the  total 
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flown  sorties  were  divided  by  the  total  planned  sorties.  The  SMG  was  the  only  MOE  that 
only  needed  to  be  summed  over  the  total  number  of  replication  days. 

3.4.  Parsing  THUNDER  reports 

The  format  for  the  reports  in  reports,  std  is  good  for  a  single  printout,  but  it  is  not 
a  good  format  for  manipulating  the  data,  reports. std  is  a  text  file.  This  means,  that  the 
format  of  the  file  is  for  a  nice  display,  which  is  generally  not  conducive  to  data 
manipulation.  In  addition  to  the  text  fonnat,  report  CC-4  is  produced  every  12- 
simulation  hours,  while  AW-3  and  AW-5  are  reported  every  24-simulation  hours. 
Therefore,  there  are  twice  as  many  CC-4  reports  as  there  are  AW-3  or  AW-5  reports. 
Fortunately,  the  reports  are  relatively  unifonn  and  can  be  manipulated  with  a  parsing 
script.  The  following  chart  describes  the  flow  of  the  parsing  script. 
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Figure  3.1:  Parsing  Script  Flow  Chart 


Using  a  combination  of  Microsoft  Excel,  Microsoft  Access  and  VBA,  the 
reports.std  is  transformed  from  an  ungainly  text  file  into  easy  to  manipulate  Access 
tables.  The  tables  produced  are  easily  combined  into  a  table  that  is  readily  transferable  to 
software  with  regression  capabilities  such  as  JMP  or  MATLAB.  THUNDER  has  been 
modified  to  produce  a  file  named  ear ltally.txt  that  contains  the  control  variate 
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information  and  earltime.txt  that  contains  the  start  and  ending  times  for  each  replication. 
A  small  example  of  the  earltally.txt  file  is  located  below. 


160, 

7, 

o., 

o., 

0, 

104, 

11, 

.1, 

.33, 

0, 

105, 

11, 

.1, 

.33, 

0, 

71, 

9, 

o., 

o., 

0, 

47, 

9, 

o., 

o., 

0, 

70, 

9, 

o., 

o., 

0, 

114, 

9, 

o., 

o., 

0, 

-.00986138176387563802,  210 
45.43071582689116816311,  210 
43.01011811890460556639,  210 
601.25462303729726443179,  1179 
975.32737950515002722795,  1955 
258.31346595147626210575,  535 
4523.83226559217987983174,  9025 


Figure  3.2:  Example  of  earltally.txt  File 


The  earltally.txt  file  is  a  comma  separated  text  file.  This  means,  that  the  file  contains 


data  separated  into  columns  by  commas.  It  has  a  total  of  7  columns.  The  first  column 


indicates  which  random  variate  the  draw  is  from.  The  second  column  indicates  what 


distribution  the  draw  is  from.  Table  3.2  decodes  all  13  of  the  distributions  used  by 
THUNDER.  The  third,  fourth,  and  fifth  columns  are  parameters  1,  2,  and  3  respectively, 
of  the  distribution.  The  sixth  column  is  the  sum  of  the  responses  for  the  specific  random 
variate  draw.  The  final  column  is  a  count  of  the  random  variate  draws. 

Table  3.2:  Random  Distribution  Decodes 


code 

decode 

1 

Beta 

2 

Binomial 

3 

Erlang 

4 

Exponential 

5 

Gamma 

6 

Logorithmic  Normal 

7 

Normal 

8 

Poisson 

9 

Random 

10 

Triangle 

11 

Uniform 

12 

Weibull 

13 

Randi 

The  diagram  in  Figure  3.1  shows  that  the  two  output  files,  earltally.txt  and 
reports. std,  are  handled  separately  and  could  be  perfonned  in  parallel  to  speed  up  post 
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processing.  Starting  with  the  earltally.txt  file,  all  replications  are  first  loaded  directly  into 
Access.  Then,  Access  singles  out  the  random  variates  with  only  one  entry  per 
replication.  Some  random  variates  have  distributions  with  parameters  that  change 
through  out  a  THUNDER  replication  and  therefore  would  have  multiple  entries  in  the 
earltally.txt  file.  Next,  Access  then  filters  out  the  random  variates  that  were  not  used  in 
all  replications.  Access  then,  calculates  the  expected  value  for  each  of  the  remaining  data 
point  and  then  subtracts  the  expected  value  from  its  corresponding  data  point.  Finally, 
Access  eliminates  the  resulting  random  variates  that  are  zero  or  extremely  close  to  zero. 
The  final  product  is  a  table  of  random  variates  with  deviations  from  the  expected  value 
that  can  be  used  as  potential  control  variates.  The  table  below  has  the  random  variates 


labeled  across  the  top  and  identifies  each  replication  as  a  row. 


Table  3.3:  Example  of  earltally.txt  Final  Product 


REPLICATION 

Total  Of 

COUNT 

RESPONSE 

1 

125583 

0.002579423 

-0.000329738 

-0.000770258 

-0.000232268 

0.004212851 

2 

132150 

-0.009506824 

-0.003220924 

-8.88252E-05 

0.00125997 

0.003561292 

3 

133485 

0.001791816 

0.001023826 

4 

125951 

0.003829476 

0.001651921 

0.000130294 

0.000752148 

5 

133589 

0.00681057 

0.00051887 

0.000205273 

0.000267634 

6 

122488 

0.000387382 

0.001067072 

0.00013618 

0.000565525 

j  0.01 0540051 1 

7 

122536 

-9.22039E-05 

filtlitUlililcldi 

yrnmnami 

8 

123779 

0.003961604 

0.000277925 

0.000663163 

9 

134503 

0.004555212 

1  0.0001 14831 1 

0.000903415 

[  0.001249704| 

0.007879616 

The  file  reports.std  undertakes  a  more  convoluted  path.  First  Access  deletes  all  of 


the  old  output  tables  and  opens  a  template  Excel  file  called  reportsifter.xls.  This  Excel 


file  contains  macros  and  sheets  designed  to  decode  the  three  reports  contained  in 


reports.std.  Then  Excel  opens  up  the  reports.std  file  and  separates  each  report  page  into 


its  own  Excel  worksheet.  After  separating  the  report  pages,  Excel  looks  through  each 


worksheet  for  report  AW-3  and  combines  important  data  from  AW-3  onto  a  new 
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worksheet.  After  all  the  AW-3  reports  are  found.  Excel  repeats  the  combining  process 
for  reports  AW-5  and  CC-4.  It  then  saves  the  three  combined  worksheets  into  a  new 
Excel  workbook.  Excel  then  repeats  the  entire  process  for  each  replication,  until  all  of 
the  reports. std  fdes  in  the  directory  have  been  loaded  and  labeled  with  its  corresponding 
replication.  Finally,  Access  looks  in  the  directory  and  transfers  all  three  worksheets, 
AW-3,  AW-5,  and  CC-4,  from  each  of  the  new  Excel  workbooks  into  Access  and 


combines  them.  A  partial  example  of  the  AW-3  finished  sheet  is  below. 


Table  3.4:  Example  of  AW-3  Final  Product 


REPLICATION 

DAY 

SIDE 

ID 

TYPE 

AIRCRAFT 

NAME 

NUMBER 

OF 

SORTIES 

LOSSES 

AIR  TO  AIR 

LOSSES 

SURFACE 

TO  AIR 

LOSSES 

GROUND 

LOSSES 

OTHER 

1 

4:0000 

BLUE 

F-16 

150 

0 

1 

0 

0 

1 

4:0000 

BLUE 

ABL 

22 

0 

1 

0 

0 

1 

4:0000 

BLUE 

C-130H 

4 

0 

0 

0 

0 

1 

4:0000 

BLUE 

1  1 01 3| 

C-17 

2 

0 

0 

0 

0 

1 

4:0000 

BLUE 

djjg 

KC-135 

58 

0 

0 

0 

0 

1 

4:0000 

BLUE 

E-8 

4 

0 

0 

0 

0 

1 

4:0000 

BLUE 

E-3 

6 

0 

0 

0 

0 

1 

4:0000 

BLUE 

F-14 

108 

1 

0 

0 

0 

1 

4:0000 

BLUE 

A-6 

101 

0 

5 

0 

0 

1 

4:0000 

BLUE 

F/A-18 

48 

0 

0 

0 

0 

1 

4:0000 

BLUE 

AV-8B 

66 

0 

0 

0 

0 

1 

4:0000 

BLUE 

|  1005| 

F-15 

296 

0 

0 

0 

0 

1 

3:0000 

BLUE 

Hjjg 

KC-135 

55 

0 

0 

0 

0 

1 

4:0000 

BLUE 

RF-4 

44 

1 

1 

0 

0 

1 

4:0000 

RED 

SU-25 

50 

5 

3 

0 

0 

1 

4:0000 

BLUE 

A-10 

254 

3 

1 

0 

0 

1 

3:0000 

RED 

SCUD 

43 

3 

0 

2 

0 

1 

3:0000 

RED 

MAINSTAY 

3 

0 

0 

0 

0 

1 

3:0000 

RED 

SU-25 

61 

5 

10 

0 

0 

1 

3:0000 

RED 

MIG-29 

8 

0 

0 

0 

0 

1 

3:0000 

RED 

MIG-21 

96 

13 

0 

0 

0 

1 

3:0000 

RED 

MIRAGE  F-1 

46 

5 

2 

0 

0 

1 

3:0000 

RED 

MIG-23 

61 

11 

8 

0 

0 

1 

3:0000 

BLUE 

ABL 

22 

0 

0 

0 

0 

1 

3:0000 

BLUE 

C-130H 

2 

0 

0 

0 

0 

When  the  scripts  are  finished,  the  Access  database  contains  four  output  tables. 


They  are  AW-3,  AW-5,  CC-4,  and  the  potential  control  variates.  Using  Access,  the 


report  tables  can  be  modified  to  obtain  the  selected  MOEs.  Then  the  selected  MOEs  can 
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be  combined  with  the  potential  control  variates  to  create  regression  ready  tables.  A  copy 
of  the  scripts  outlined  above  is  available  from  the  Center  for  Operations  Analysis  (COA) 
at  AFIT.  Also,  a  list  of  the  modified  files  is  located  in  Appendix  (C). 

3.5.  Control  Variates 

The  method  of  Control  Variates  relies  on  the  output  of  a  random  variate  where  the 
expected  value  is  known.  In  simulations,  the  random  variates  generated  throughout  a 
replication  are  effectively  random  and  have  a  known  expected  value.  Therefore,  all  the 
random  variates  generated  in  a  simulation  are  potential  control  variates  and  should  be 
identified  and  output  from  the  simulation.  When  creating  an  output  file  of  random 
variates,  the  variates  should  be  labeled  and  produced  with  the  label,  type,  parameters,  and 
result.  Therefore,  the  variates  can  be  identified  by  the  labels,  the  mean  can  be  calculated 
from  the  result,  and  the  expected  value  can  be  determined  by  the  type  and  parameters. 

In  the  late  1980’s,  a  very  limited  version  of  the  CRN  technique  was  added  to 
THUNDER.  Reviewing  this  work,  245  of  the  254  points  where  random  variates  are 
drawn  were  identified  by  following  the  nomenclature  of  CRN  streams.  The  control.dat 
file,  see  Appendix  (D),  contains  the  list  of  random  inputs  and  allows  the  user  to  assign 
random  number  streams  to  each  of  the  random  inputs.  The  nomenclature  of  the  random 
input  identifies  which  module  the  random  input  is  from  and  also  labels  for  more  than  one 
random  input  from  a  module.  For  example,  ..AIR070.2  is  a  random  input  from  module 
AIR070.SIM  and  it  is  the  second  random  input  from  that  module. 

Since  the  random  variate  draws  had  been  identified,  the  problem  was, 
understanding  how  the  variates  are  drawn,  what  they  are  used  for,  and  how  to  output 
them.  There  are  two  methods  used  in  THUNDER  to  draw  a  random  variate.  The  first 
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method  is  where  a  random  variate  is  drawn  in  a  logical  expression.  An  example  is  shown 
below. 


''determine  result  of  shot 

IF  .  PK 

>  0.0 

AND 

RANDOM. F(RN. STREAM( . .AIR070. 2) )  <=  . PK 

LET 

.NUM. KILLED  =  1 

ELSE 

LET 

.NUM. KILLED  =  0 

ALWAYS 

Figure  3.3:  Logical  Expression  Example  (Original) 


The  bolded  section  in  Figure  3.3  is  the  random  variate  draw.  The  code  RANDOM.F(  )  is 
the  SIMSCRIPT  II. 5  function  for  a  uniform  (0,  1)  draw.  RN.STREAM  is  a  vector  that 
contains  the  stream  number  for  each  random  number  draw.  ..AIR070.2  is  the  label  of  the 
random  variate  draw.  To  capture  the  result,  the  random  variate  is  drawn  immediately 
prior  to  its  use  in  THUNDER  and  placed  in  a  variable  called  EARL.RESPONSE. 
Following  the  random  number  draw,  the  parameters  are  placed  in  variables  EB.P1, 
EB.P2,  and  EB.P3.  When  a  parameter  is  not  used  it  is  set  to  zero  as  in  Figure  3.4. 
EARL.RESPONSE  is  then  sent  to  an  output  routine  EARL.STORE,  along  with  the 
variate  name,  variate  type,  and  all  three  parameters.  EARL.STORE  was  specifically 
built  to  organize  the  random  variate  output  for  ease  of  use.  Creating  a  single  routine  for 
all  variate  output  allows  for  the  ease  of  changing  the  output  format  if  needed.  After 
calling  the  output  routine,  THUNDER  uses  EARL.RESPONSE  in  place  of  the  original 
random  variate  draw.  Figure  3.4  is  the  modified  version  of  logical  expression  in  Figure 
3.3.  The  added  code  to  export  the  results  is  in  bold. 
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''determine  result  of  shot 

' ' ********************* 

' ' *  Earl  stuff  * 

' ' ********************* 

EARL . RESPONSE  =  RANDOM . F (RN . STREAM (. .AIR070 . 2 ) ) 

EB.P1  =  0.0 
EB.P2  =  0.0 
EB.P3  =  0.0 

CALL  EARL. STORE 
GIVEN 

. . AIR070 . 2 , 

. .RANDOM, 

EARL . RESPONSE , 

EB.P1, 

EB . P2 , 

EB.P3 

' ' *  END  STUFF 

IF  .PK  >  0.0 

AND  EARL . RESPONSE  <=  . PK 

LET  .NUM. KILLED  =  1 
ELSE 

LET  .NUM. KILLED  =  0 

_ ALWAYS _ 

Figure  3.4:  Logical  Expression  Example  (Modified) 

The  second  method  to  draw  a  random  variate  in  THUNDER  is  through  a 
subroutine.  The  subroutine  is  called  because  for  the  specific  random  variate,  the  user  can 
set  the  variate  distribution  in  an  input  file.  Therefore  the  subroutine  identifies  the 
parameters  and  which  distributions  are  needed  and  then  draws  from  that  distribution.  The 
results  from  these  random  variate  draws  are  returned  from  the  subroutine  and  stored  in  a 
variable  to  be  used  by  THUNDER.  The  example  code  in  Figure  3.5  demonstrates  a 
random  variate  subroutine  call.  The  name  of  subroutine  called  is 
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UTL.720.CALCULATE.USER.FUNCTION.  As  in  the  first  method,  it  has  its  own 
random  variate  identifier,  which  is  ..AIR070.1.  The  subroutine  returns  the  value  and 
places  it  in  .SHOT.DURATION.SEC,  which  is  then  used  in  the  lines  following  the 
subroutine  call. 


IF  . DTBM . WPN . TYPE  =  . . AIRBORNE . LASER 

OR  .DTBM. WPN. TYPE  =  .. SPACE . BASED . LASER 

CALL  UTL .720. CALCULATE . USER . FUNCTION 
GIVEN 

DWP .SHOT. DURAT I ON . FN ( . DWP)  , 

RN . STREAM ( . .AIR070.1) 

YIELDING 

. SHOT . DURATION . SEC 

LET  .SHOT.DURATION.SEC  = 

MAX. F(. SHOT. DURATION. SEC,  0.0) 

LET  .TOTAL. ELAPSED. SECONDS 

=  . DETECT . SECONDS . DELAY  +  .SHOT.DURATION.SEC 
LET  . INTERCEPT . FLAG  =  ..YES 

_ ELSE _ 

Figure  3.5:  Random  Variate  Subroutine  Call  Example  (Original) 

To  collect  the  results,  code  is  added  after  the  subroutine  returns  the  random  variate,  but 


before  it  is  used.  The  variate  result,  along  with  the  variate  name,  type  and  parameters  are 
sent  to  the  same  EARL. STORE  as  in  the  first  method.  Below  is  the  modified  version  of 


Figure  3.5. 
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IF  . DTBM . WPN . TYPE  =  .. AIRBORNE . LASER 

OR  .DTBM. WPN. TYPE  =  .. SPACE . BASED . LASER 

CALL  UTL .720. CALCULATE . USER . FUNCTION 
GIVEN 

DWP . SHOT . DURATION . FN ( . DWP) , 

RN. STREAM ( . .AIR070.1) 

YIELDING 

. SHOT . DURATION . SEC 

' ' ******************* 

' ' *  Earl  stuff  * 

' ' ******************* 

EARL . RESPONSE  =  . SHOT . DURATION . SEC 

EB.P1  =  UF.PARM1 (DWP. SHOT. DURATION. FN ( .DWP) ) 

EB.P2  =  UF.PARM2 (DWP. SHOT. DURATION. FN ( .DWP) ) 

EB.P3  =  0.0 

CALL  EARL. STORE 
GIVEN 

. . AIR070 . 1 , 

UF . FUNCTION (DWP . SHOT . DURATION . FN ( . DWP) ) , 
EARL . RESPONSE , 

EB . PI , 

EB . P2 , 

EB.P3 

' ' *  END  STUFF 

LET  .SHOT. DURATION. SEC  = 

MAX. F (. SHOT . DURATION. SEC,  0.0) 

LET  .TOTAL. ELAPSED. SECONDS 

=  .DETECT. SECONDS. DELAY  +  . SHOT . DURATION . SEC 
LET  . INTERCEPT . FLAG  =  ..YES 

_ ELSE _ 

Figure  3.6:  Random  Variate  Subroutine  Call  Example  (Modified) 

Note  the  code  added  to  THUNDER  is  identified  at  the  beginning  of  a  section  with  Earl 


stuff  and  at  an  ending  of  a  section  with  END  STUFF.  For  a  full  comparison.  Appendix 
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(E)  and  Appendix  (F)  contain  the  full  unmodified  and  modified  code  respectively  of 
Figure  3.3  through  Figure  3.6. 

A  single  five  day  run  of  the  ME  scenario  in  THUNDER  can  result  in  over 
500,000  random  variate  draws.  Outputting  every  random  variate  can  create  a  very  large 
output  file.  To  save  time  and  space,  the  random  variates  are  stored  as  a  table  within 
THFTNDER.  Therefore,  instead  of  storing  the  responses  individually,  the  table  keeps  a 
running  sum  of  the  responses  for  repeated  random  variate  draws  and  keeps  track  of  the 
number  of  draws.  THUNDER  outputs  this  matrix  in  to  the  earltally.txt  file  upon 
completion  of  a  replication.  Figure  3.2  is  an  example  of  how  the  data  is  stored. 

Using  the  script  outlined  above,  earltally.txt  can  be  combined  with  MOEs  and 
displayed  as  a  table  that  can  be  entered  into  a  regression  software  package.  Regression 
requires  the  amount  of  replications  to  equal  or  exceed  the  final  number  of  control 
variates.  Since  there  is  a  possibility  of  having  254  control  variates,  at  least  254 
replications  should  be  performed  to  allow  for  the  maximum  number  of  control  variates. 
Stepwise  regression  is  preferred  to  single  out  the  control  variates  that  have  statistically 
significant  correlation  with  the  selected  MOE.  There  are  many  ways  to  perform  a 
stepwise  regression;  the  stepwisefit  function  in  Matlab  was  used  for  this  research.  Three 
hundred  replications  of  the  scenario  were  used.  Then  a  matrix  of  potential  controls  and 
MOEs  were  imported  into  a  Matlab  script.  The  matrix  of  potential  controls  contained  the 
potential  control  variates  left  from  the  parsing  script  minus  the  known  mean.  Then 
stepwisefit  with  a  parameter  of  max  p-value  to  enter  of  0.06  was  used  to  identify  the 
controls  that  are  statistically  significant  to  each  MOE.  After  the  controls  were  identified 
for  each  MOE,  the  MOEs  are  regressed  with  their  corresponding  controls  and  output  into 
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an  Excel  workbook.  A  halfwidth  for  each  MOE  can  be  calculated  with  the  known 
controls.  This  new  halfwidth  should  be  smaller  than  a  halfwidth  calculated  from 
replications. 

3.6.  Common  Random  Numbers 

As  stated  before,  THUNDER  had  previously  been  coded  for  CRNs.  The  coding 
identifies  the  points  where  random  numbers  are  drawn.  Therefore,  potential 
synchronization  of  the  random  number  draws  was  already  in  place.  Also,  the  control.dat 
file  contains  a  method  of  assigning  points  where  random  variates  are  drawn  from  a 
specific  random  number  stream.  The  problem  with  using  CRNs  with  THUNDER  is  that 
the  random  number  generator  (RNG)  only  allows  for  10  streams,  while  there  are  254 
random  inputs  where  streams  could  be  used.  To  solve  this  problem,  a  new  random 
number  generator  was  added  to  THUNDER.  The  random  number  generator  proposed  by 
Pierre  L’Ecuyer  (18)  has  1.8  x  10  streams,  each  of  length  1.7  x  10  elements.  To  put 
this  in  perspective,  a  standard  software  package  RNG  normally  does  not  have  streams 
and  its  entire  length  is  generally  between  1.6  x  107  and  2.8  x  1014  elements.  As  an  added 
benefit,  the  random  number  generator  is  already  coded  in  C  with  a  support  package. 
SIMSCRIPT  II. 5  will  readily  compile  routines  programmed  in  C  and  incorporates 
methods  to  call  the  compiled  routines.  Therefore,  the  proposed  random  number 
generation  package  could  be  compiled  with  THUNDER. 

The  current  version  of  THUNDER  tallies  how  many  random  number  calls  are 
made  during  a  simulation  run.  It  does  this  by  rerouting  a  random  number  draw  to  a 
subroutine  that  increments  a  counter,  draws  a  uniform  [0,  1]  random  number,  and  returns 
the  unifonn  [0,  1]  random  number.  Therefore  to  direct  THUNDER  to  use  the  new  RNG 
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all  that  is  needed  to  be  done  is  replace  the  uniform  [0,  1]  random  number  draw  from  the 
standard  function  with  a  call  to  the  new  random  number  routines. 

Not  all  random  inputs  are  uniform  [0,  1]  random  numbers.  When  THUNDER 
was  coded  to  tally  the  number  of  random  number  draws,  a  series  of  subroutines  were 
built  to  transfonn  uniform  [0,  1]  random  numbers  into  random  variates  from  specified 
distributions  with  given  parameters. 

A  new  RNG  designator  has  been  placed  in  the  control. dat file.  Appendix  (D), 
used  to  run  a  THUNDER  replication.  If  this  is  set  to  YES,  THUNDER  will  use  the  new 
RNG.  When  the  new  RNG  is  used,  it  reads  a  new  seed  file  that  contains  the  number  of 
the  first  stream.  If  CRN  designator  in  control.dat  is  also  set  to  YES,  THUNDER  will 
then  create  a  separate  stream  for  each  of  the  254  points  where  random  variates  are  drawn. 
At  the  end  of  a  replication,  THUNDER  records  the  next  starting  seed  in  the  seed  file  for 
each  replication. 

3.7.  Antithetic  Variates 

Antithetic  variates,  like  CRN,  rely  on  synchronization.  Since  THUNDER  has 
been  setup  with  CRN,  AV  can  utilize  the  synchronization.  To  implement  AV,  a 
replication  of  the  simulation  should  be  run  normally  with  the  new  RNG  and  CRN  in 
control.dat  set  to  YES.  Then  a  second  replication  of  the  simulation  with  the  same 
streams  should  be  run  with  the  new  RNG,  CRN,  and  AV  in  control.dat  set  to  YES.  This 
will  cause  the  simulation  to  be  run  with  all  the  random  numbers  to  be  1  -  the  random 
numbers  of  the  previous  replication.  Then  the  two  replications  are  averaged  to  obtain  the 
variance  reduction.  This  may  seem  tedious,  but  if  all  the  random  input  are  synchronized 
as  in  CRN,  then  a  set  of  replications  can  be  run  normally  and  then  followed  by  a  set  of 
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antithetic  runs  synchronized  to  the  previous  runs.  Upon  completion,  the  MOEs  can  be 
averaged. 

The  new  RNG  package  compiled  with  THUNDER  comes  complete  with  the 
ability  to  use  antithetic  variates.  The  way  this  is  implemented  is  by  designating  a  stream 
as  antithetic  upon  creation.  The  modified  control.dat  file  has  an  antithetic  designator. 
Setting  the  designator  to  YES,  will  assign  all  random  number  streams  to  antithetic.  In 
addition,  to  perform  AV  effectively,  the  common  random  number  designator  should  also 
be  set  to  YES  for  both  the  nonnal  and  antithetic  replications. 

3.8.  Integrated  Control  and  Antithetic  Variates 


The  third  method  outlined  by  Kwon  and  Tew  (15)  was  used  to  analyze  the 
variance  reduction  gained  by  integrating  Control  Variates  and  Antithetic  Variates.  Their 
method  is  relatively  simple.  First,  run  a  simulation  as  you  would  for  AVs  with  n 
replications,  while  producing  the  random  variate  output  file.  Then,  combine  each  of  the 


n/ 

/2 


antithetic  pairs  and  average  them.  This  includes  averaging  both  corresponding 


random  variates  and  corresponding  MOEs.  The  result  is  independent  antithetic  pairs. 


Then,  use  the  antithetic  pairs  to  perform  a  normal  Control  Variate  analysis. 

3.9.  Integrated  Control  Variates  and  Common  Random  Numbers 

To  analyze  the  variance  reduction  by  integrating  Control  Variates  and  Common 
Random  Numbers  a  modification  of  the  method  employed  by  Yang  and  Nelson  (44)  was 
used  because  Yang  and  Nelson’s  multiple  comparisons  relies  on  identical  control  variates 
and  THUNDER  does  not  produce  identical  control  variates,  even  with  synchronization. 


The  modified  method  is  the  application  of  the  control  variate  estimators  to  develop 
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paired-t  confidence  intervals.  First,  independent  THUNDER  runs  were  perfonned  using 
the  base  model  database  and  the  modified  model  database.  These  runs  were  used  as  a 
base  paired-t  confidence  interval  and  to  identify  the  random  variates  that  are  significantly 
correlated  with  the  MOEs.  These  random  variates  were  then  synchronized  while  the  rest 
were  left  independent  for  the  analysis  runs.  Then,  the  control  variate  estimators 
identified  in  the  base  runns  estimate  the  response  and  are  used  in  the  paired-t  confidence 
intervals.  The  nonnal  distribution  is  used  instead  of  the  studentized  t  distribution  of 
calculating  the  halfwidth.  This  is  because  the  studentized-t  distribution  resembles  the 
nonnal  distribution  when  the  degrees  of  freedom  become  greater  than  150. 

3,10.  Divide  and  Conquer  and  the  Major  Shared  Resource  Center 

The  first  2400  THUNDER  replications  were  performed  on  the  ASC  MSRC  HPC 
resources,  while  the  last  were  performed  on  the  AFIT  Linux  cluster.  To  perfonn  the 
different  types  of  analysis  for  this  research,  THUNDER  needed  to  be  run  under  several 
different  settings.  To  organize  and  set  up  all  of  the  replications  with  the  different  settings 
can  be  cumbersome.  To  make  this  practical,  the  ASC  MSRC  Divide  and  Conquer  (DAC) 
web  interface  was  used.  The  following  diagram  depicts  the  DAC  process. 
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Figure  3.7:  Divide  and  Conquer  Flow  Diagram  (29) 


The  DAC  has  a  graphical  user  interface  (GUI)  that  loads  a  simulation  template  and 
allows  the  user  to  set  up  and  modify  select  parameters  for  analysis.  It  even  allows  the 
user  to  select  multiple  levels  of  a  parameter.  After  the  parameters  are  set,  the  DAC  web 
interface  will  automatically  organize  the  runs  and  create  scripts  to  perform  the  runs. 

These  scripts  are  compressed  into  a  .tar  file  and  delivered  to  the  proper  HPC  resource, 
unpacked  and  run. 

3.11.  Analysis 

To  analyze  the  data,  the  confidence  interval  halfwidths  of  the  base  case,  no  VRT, 
are  compared  with  the  confidence  interval  halfwidths  generated  by  the  variance  reduction 
technique.  Access  databases  were  built  to  manipulate  the  data  into  the  MOEs  for  each 
technique  and  then  exported  to  Excel  for  ease  of  comparison.  Then  the  variance 
reduction  is  calculated  by  dividing  the  difference  between  the  base  halfwidth  and  the 
VRT  halfwith  by  the  base  halfwidth. 


3.19 


A  paired-t  confidence  interval  is  used,  to  determine  if  a  pair  of  values  is  the  same. 
This  test  determines  the  pair  of  values  to  be  equal  if  the  confidence  interval  includes  zero. 


All  statistical  tests  were  calculated  at  the  a=  0.05  level. 


Since  there  were  three  types  of  variance  reduction  techniques  to  analyze,  different 
experiment  settings  were  needed.  A  total  of  eight  different  run  settings  were  identified 
for  all  of  the  experiments,  with  300  replications  at  each  setting  for  2400  total  THUNDER 
replications.  The  following  table  identifies  the  different  setting  used  for  the  VRT 
analysis. 


Table  3.5:  Experiment  Settings 


RNG 

SYNCHRONIZED 

ANTITHETIC 

COLLECT  CV 

CONFIGURATION 

SET 

(NEW/OLD) 

(YES/NO) 

(YES/NO) 

(YES/NO) 

(ORIGINAL/MODIFIED) 

A 

OLD 

NO 

NO 

YES 

ORIGINAL 

B 

NEW 

NO 

NO 

NO 

ORIGINAL 

C 

NEW 

NO 

NO 

YES 

MODIFIED 

D 

NEW 

YES 

NO 

YES 

ORIGINAL 

E 

NEW 

YES 

NO 

YES 

MODIFIED 

F 

NEW 

YES 

YES 

YES 

ORIGINAL 

G 

OLD 

NO 

NO 

NO 

ORIGINAL 

H 

NEW 

YES 

NO 

NO 

ORIGINAL 

NEW 

NO* 

NO 

YES 

ORIGINAL 

O 

NEW 

NO* 

NO 

YES 

MODIFIED 

S 

NEW 

YES* 

NO 

YES 

ORIGINAL 

T 

NEW 

YES* 

NO 

YES 

MODIFIED 

RNG  identifies  use  of  the  current  (OLD)  or  MRG32k3a  (NEW)  random  number 


generator.  SYNCHRONIZED  indicates  wheter  all  of  the  random  inputs  are 
synchronized  or  not.  The  values  of  (YES*)  indicates  that  only  some  of  the  random 
variates  are  synchronized  and  the  values  (NO*)  indicates  a  different  stream  than  1  was 
used  to  generate  the  independent  random  variates.  ANTITHETIC  identifies  if  the 
random  number  draws  are  drawn  normally  (NO)  or  using  1  -U  (YES).  COLLECT  CV 
identifies  whether  the  replications  output  a  controls  file.  CONFIGURATION  inidicates 
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the  use  of  the  standard  ME  model  database  (ORIGINAL)  or  the  use  of  the  ME  model 
database  with  the  209  F-l  1  l’s  eliminated  (MODIFIED). 
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4.  Analysis 


This  section  contains  results  and  examples  of  analysis  performed  to  detennine  the 
effectiveness  of  the  variance  reduction  techniques  applied  to  THUNDER.  The  first  three 
sections  cover  the  results  from  applying  the  control  variate  technique,  common  random 
numbers,  and  antithetic  variates  respectively.  The  fourth  section  details  the  combination 
of  both  control  variates  and  antithetic  variates.  The  fifth  sections  reports  on  the 
combination  of  control  variates  with  common  random  numbers.  The  last  section  covers 
the  comparison  between  the  new  random  number  generator  and  the  original  random 
number  generator. 

4.1.  Control  Variates 

A  total  of  600  replications  of  the  ME  scenario  were  performed  to  analyze  the 
effectiveness  of  the  control  variate  technique.  Table  4.1  details  the  experiment  settings; 
remember  that  each  set  consists  of  300  replications.  All  of  the  replications  used  the 
original  random  number  generator,  with  no  common  random  numbers,  with  no  antithetic 
variates  and  used  the  original  configuration  of  the  ME  scenario.  The  first  300 
replications,  set  A,  generated  control  variate  information.  They  were  used  to  determine 
the  halfwidth  for  each  MOE  with  and  without  control  variates.  The  second  300 
replications,  set  G,  were  generated  without  control  variates.  They  were  used,  in 
conjunction  with  the  set  A,  for  comparing  the  difference  in  speed  between  generating 
control  variates  and  not  generating  control  variates. 
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Table  4.1:  Control  Variate  Experiment  Settings 


RNG 

SYNCHRONIZED 

ANTITHETIC 

COLLECT  CV 

CONFIGURATION 

SET 

(NEW/OLD) 

(YES/NO) 

(YES/NO) 

(YES/NO) 

(ORIGINAL/MODIFIED) 

A 

OLD 

NO 

NO 

YES 

ORIGINAL 

G 

OLD 

NO 

NO 

NO 

ORIGINAL 

The  following  analysis  compares  the  halfwidths  generated  by  standard  replication 


and  control  variates.  The  second  subsection  estimates  the  number  of  extra  replications 


needed  to  achieve  a  similar  reduction  as  the  reduction  achieved  through  control  variates. 


The  following  subsection  demonstrates  how  control  variates  may  be  used  in  gaining 


insight  into  the  workings  of  the  simulation.  The  last  subsection  provides  an  analysis  of 


the  simulation  speed,  comparing  time  needed  to  produce  a  control  variate  output  fde  and 


the  time  needed  to  run  a  simulation  without  producing  a  control  variate  file. 


4.1.1.  Replications  vs.  Variance  Reduction  Technique. 


The  ME  scenario  was  replicated  300  times,  generating  the  control  variate  file  and 


in  turn,  creating  a  potential  control  variate  table.  The  control  variate  table,  along  with  the 


response  table  was  processed  through  the  control  variate  script  to  produce  the  control 


variate  halfwidths.  Then  the  halfwidths  for  each  MOE  were  calculated  at  the  a=  0.05 


level  and  compared  to  the  control  variate  halfwidths.  The  results  are  below. 
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Table  4.2:  Control  Variate  Halfwidth  Reduction 


MOE 

REPLIC 

MEAN 

NATION 

HALFWIDTH 

CONTROL 

MEAN 

.  VARIATE 

HALFWIDTH 

PERCENT 

CHANGE 

A-10LPS 

0.004928615 

0.000271598 

0.004947585 

0.000270219 

-0.51% 

A-6  LPS 

0.018843645 

0.001023321 

0.018829341 

0.001014611 

-0.85% 

BLUE  FPS 

0.581540025 

0.002144585 

0.581287551 

0.002034784 

-5.12% 

BLUE  LPS 

0.010229418 

0.00015922 

0.010225498 

0.000157836 

-0.87% 

BLUE  SMG 

34487.47 

81.58752506 

34501.15753 

79.42601286 

-2.65% 

DHAHRAN  FPS 

0.374067144 

0.007240438 

0.373871063 

0.007070694 

-2.34% 

F-111  LPS 

0.017016367 

0.000422881 

0.016995175 

0.00041728 

-1.32% 

F-15LPS 

0.00642758 

0.000296031 

0.006433521 

0.000293957 

-0.70% 

FA  18  FPS  total 

0.232338281 

0.003721562 

0.232495993 

0.003661179 

-1.62% 

FA  18  FPS  DCA 

0.043787767 

0.001197699 

0.043683747 

0.001181571 

-1.35% 

MIG-23  LPS 

0.253296028 

0.006221384 

0.252744641 

0.00619634 

-0.40% 

MIG-29  LPS 

0.195555155 

0.007370947 

0.19570513 

0.007264416 

-1.45% 

MIRAGE  FPS  total 

0.598071397 

0.004573737 

0.598463965 

0.004517323 

-1.23% 

MIRAGE  FPS  INT 

0.702633484 

0.005673337 

0.703846015 

0.005482963 

-3.36% 

MIRAGE  LPS 

0.274969404 

0.00452846 

0.275087315 

0.004501166 

-0.60% 

MUDAYSIS  FPS 

0.19221417 

0.005066926 

0.192327882 

0.004982654 

-1.66% 

RED  FPS 

0.391637103 

0.001710069 

0.39176041 

0.001693386 

-0.98% 

RED  LPS 

0.192105922 

0.001578977 

0.19217664 

0.001531504 

-3.01% 

RIYADH  FPS 

0.855569083 

0.001545936 

0.855668121 

0.001516207 

-1.92% 

SHAIBAH  FPS 

0.499191886 

0.006482834 

0.499297882 

0.006409984 

-1.12% 

A  reduction  in  all  the  halfwidths  is  realized.  The  reduction  may  be  small,  but  this  is 
expected  because  as  we  can  see  in  Table  4.3,  R  from  each  model  is  also  small. 
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Table  4.3:  Control  Variate  RSquared 


MOE 

RSQUARED 

ADJUSTED 

RSQUARED 

A10LPS 

0.017990681 

0.017990681 

A6  LPS 

0.026694585 

0.02342846 

BLUE  FPS 

0.12506954 

0.11018977 

BLUE  LPS 

0.021109417 

0.021109417 

BLUE  SMG 

0.100168792 

0.081742215 

DHAHRAN  FPS 

0.059778386 

0.053446928 

F-111  LPS 

0.040550054 

0.034089112 

F- 15  LPS 

0.024657888 

0.021384928 

FA-18  FPS  total 

0.046169914 

0.039746816 

FA-18  FPS  DCA 

0.038015219 

0.034787082 

MIG-23  LPS 

0.025816687 

0.022547616 

MIG-29  LPS 

0.060690505 

0.051170476 

MIRAGE  FPS  total 

0.044797447 

0.038365106 

MIRAGE  FPS  INT 

0.095539229 

0.083275355 

MIRAGE  LPS 

0.016059425 

0.016059425 

MUDAYSIS  FPS 

0.052473379 

0.046092729 

RED  FPS 

0.026028565 

0.026028565 

RED  LPS 

0.06941087 

0.063144276 

RIYADH  FPS 

0.066740758 

0.05728205 

SHAIBAH  FPS 

0.029355585 

0.026098389 

A  graphical  example  of  the  variance  reduction  can  be  seen  in  the  following  examples. 


They  are  the  largest  reduction,  BLUE  Flown  per  Planned  Sortie,  and  the  smallest 


reduction,  MIG-23  Losses  per  Sortie. 


Replication 

Control  Variate 

I  I  I  I  I  I  I 

0.577  0.578  0.579  0.58  0.581  0.582  0.583  0.584  0.585 

Figure  4.1:  BLUE  Flown  per  Planned  Sortie  Confidence  Intervals 
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Replication 


Control  Variate 


0.24  0.245  0.25  0.255  0.26  0.265 


Figure  4.2:  MIG-23  Losses  per  Sortie  Confidence  Intervals 

There  is  a  shift  in  the  means  between  the  replication  and  control  varates.  This  is 
because  the  calculations  for  the  replication  mean  is  calculated  by  using  the  arithmetic 
average  and  the  CV  mean  is  calculated  from  regression  using  least  squares  estimates. 
Using  the  means  from  Table  4.2,  the  following  tabe  was  genereated  and  the  absolute 
percent  change  was  calculated. 


Table  4.4:  Absolute  Percent  Change  In  Means 


MOE 

REPLICATION 

MEAN 

CV 

MEAN 

PERCENT 

CHANGE 

A-10  LPS 

0.004928615 

0.004947585 

0.38% 

A-6  LPS 

0.018843645 

0.018829341 

0.08% 

BLUE  FPS 

0.581540025 

0.581287551 

0.04% 

BLUE  LPS 

0.010229418 

0.010225498 

0.04% 

BLUE  SMG 

34487.47 

34501.15753 

0.04% 

DHAHRAN  FPS 

0.374067144 

0.373871063 

0.05% 

F-111  LPS 

0.017016367 

0.016995175 

0.12% 

F- 15  LPS 

0.00642758 

0.006433521 

0.09% 

FA-18  FPS  total 

0.232338281 

0.232495993 

0.07% 

FA-18  FPS  DCA 

0.043787767 

0.043683747 

0.24% 

MIG-23  LPS 

0.253296028 

0.252744641 

0.22% 

MIG-29  LPS 

0.195555155 

0.19570513 

0.08% 

MIRAGE  FPS  total 

0.598071397 

0.598463965 

0.07% 

MIRAGE  FPS  INT 

0.702633484 

0.703846015 

0.17% 

MIRAGE  LPS 

0.274969404 

0.275087315 

0.04% 

MUDAYSIS  FPS 

0.19221417 

0.192327882 

0.06% 

RED  FPS 

0.391637103 

0.39176041 

0.03% 

RED  LPS 

0.192105922 

0.19217664 

0.04% 

RIYADH  FPS 

0.855569083 

0.855668121 

0.01% 

SHAIBAH  FPS 

0.499191886 

0.499297882 

0.02% 
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As  can  be  seen  above,  all  of  the  absolute  percent  changes  in  means  are  positive,  therefore 
the  shift  in  means  is  relatively  insignificant. 

4.1,2.  Variance  Reduction  Technique  to  Number  of  Replications. 

Another  way  of  looking  at  the  reduction  gained  by  using  control  variates  it  to 
calculate  the  number  of  replications  needed  to  achieve  the  control  variate  halfwidth.  To 
do  this,  the  halfwidth  calculation  is  iterated  by  incrementing  the  number  of  replications  in 
the  equation  until  the  halfwidth  is  smaller  than  the  control  variate  halfwidth.  A  major 


assumption  to  this  method  is  the  variance  remains  constant.  The  following  table  displays 
the  number  of  replications  needed  to  achieve  the  variance  reduction. 

Table  4.5:  Number  of  Replications  Required  to  Achieve  Control  Variate  Reduction 


MOE 

REPLICATIONS 

CHANGE  IN 

REPLICATIONS 

PERCENT 

CHANGE 

A-10LPS 

304 

4 

1 .33% 

A-6  LPS 

306 

6 

2.00% 

BLUE  FPS 

333 

33 

1 1 .00% 

BLUE  LPS 

306 

6 

2.00% 

BLUE  SMG 

317 

17 

5.67% 

DHAHRAN  FPS 

315 

15 

5.00% 

F-111  LPS 

309 

9 

3.00% 

F- 15  LPS 

305 

5 

1 .67% 

FA-18  FPS  total 

310 

10 

3.33% 

FA- 18  FPS  DCA 

309 

9 

3.00% 

MIG-23  LPS 

303 

3 

1 .00% 

MIG-29  LPS 

309 

9 

3.00% 

MIRAGE  FPS  total 

308 

8 

2.67% 

MIRAGE  FPS  INT 

321 

21 

7.00% 

MIRAGE  LPS 

304 

4 

1 .33% 

MUDAYSIS  FPS 

311 

11 

3.67% 

RED  FPS 

306 

6 

2.00% 

RED  LPS 

319 

19 

6.33% 

RIYADH  FPS 

312 

12 

4.00% 

SHAIBAH  FPS 

307 

7 

2.33% 

4.6 


4.1.3.  Control  Variate  Insight. 

The  method  of  control  variates  has  an  added  benefit  to  the  analyst.  It  generates  a 
series  of  random  inputs  that  are  statistically  significant  in  their  correlation  to  the  selected 
MOE.  Knowledge  about  these  random  inputs  can  give  insight  into  how  the  simulation 
works.  First,  knowing  the  weight  and  sign  of  the  coefficient  can  inform  the  analyst  how 
much  and  in  which  direction  a  shift  in  the  control  can  alter  the  selected  MOE.  The 
following  table  identifies  the  significant  controls  for  each  MOE  and  gives  the  weight  and 
sign  of  the  corresponding  control. 
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Table  4.6:  Control  Variate  Statistically  Significant  Controls 


MOE 

WEIGHT 

CONTROL 

MOE 

WEIGHT 

CONTROL 

A-10LPS 

-0.0002 

MIG-23  LPS 

-6.6526 

■■MW— 

A-6  LPS 

-0.0071 

0.1433 

-0.0759 

MIG-29  LPS 

0.6471 

BLUE  FPS 

-0.0722 

-1.4314 

-0.0659 

Wki'W'M 

-0.2311 

-0.0035 

-0.7147 

0.3282 

MIRAGE  FPS  total 

-0.3242 

46.8237 

0.5198 

0.6659 

0.0855 

BLUE  LPS 

0.0101 

MIRAGE  FPS  INT 

-3.3544 

BLUE  SMG 

-23825.1823 

7.9362 

40.5388 

_ AIR060_2 

0.0101 

niaa.nacl 

3649.5981 

-1.0679 

■**.1*1*1 

-1408.6753 

MllMAWm 

106.4336 

-20765.0668 

MEH5EOI 

MIRAGE  LPS 

-2.3615 

B51HM 

-27766.1967 

MUDAYSIS  FPS 

-0.2438 

warn aa 

-8446.2005 

-0.0089 

magi 

DHAHRAN  FPS 

-0.1779 

0.1621 

-0.2230 

MMMMm 

RED  FPS 

0.6949 

0.9677 

RED  LPS 

0.0922 

F-111  LPS 

0.0213 

MESSmWM 

-0.0629 

WSMMM 

-0.0120 

MSSMSMM 

0.6117 

BBSEiB 

-0.1221 

MsummM 

RIYADH  FPS 

-0.2501 

F- 15  LPS 

0.0069 

-0.0395 

0.0019 

-0.0621 

FA-18  FPS  total 

0.0252 

mxiziwwm 

-0.0505 

-0.0889 

wa 

SHAIBAH  FPS 

-0.1675 

-1.3057 

SBHB 

1.3115 

FA- 18  FPS  DCA 

1.4195 

0.0006 

The  table  above  can  also  be  used  in  conjunction  with  knowledge  of  the  simulation 


code.  If  the  analyst  looks  at  how  the  controls  are  used,  he  may  be  able  to  identify 


systems  or  actions  that  affect  the  MOE.  For  example,  take  MOE  BLUE  SMG  (Square 


Miles  Gained).  The  controls  ADF1051,  AIR060  2,  AIR1012,  AIR5612,  AIR840  1, 
AIR840  2,  and  BSE400  1  are  all  significant.  By  looking  at  the  code,  the  use  of  each  of 
the  random  variates  can  be  identified  by  examination  of  the  module  purpose,  the 
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comments  in  the  code  and  the  nomenclature  of  the  interacting  variables.  The  following 
table  outlines  information  gleaned  from  the  code. 


Table  4.7:  Control  Variate  Purpose  of  Statistically  Significant  Controls 


CONTROL 

WEIGHT 

MODULEPURPOSE 

SPECIFIC  DRAW  PURPOSE 

ADF105_1 

-23825.18 

Used  to  calculate  the  flight  groups 
position,  altitude,  speed,  average 
location,  and  delivery  profile. 

Used  to  check  for  point  on  battlefield. 
(Distribution) 

AIR060_2 

40.54 

Manages  defensive  anti  tactical 
ballistic  missile  inis s ion  detections  and 
engagements. 

Used  in  setting  saturation  delay  on 
shooter.  (Distribution) 

AIR101_2 

3649.60 

Sets  up  and  air  engagement  by  creating 
and  air-to-air  engagement  for  each 
flight  to  be  used  for  storing 
computations. 

Used  in  determining  the  engagement 
probability,  (.prob.engage  <  U(0,1)) 

AIR561_2 

-1408.68 

Determines  if,  having  arrived  at  the 
target’s  estimated  coordinate,  the  flight 
group  can  find  the  target. 

Det  amines  the  success  of  a  mid¬ 
course  update.  (U(0,1)<=  .best.prob) 

AIR840_1 

-20765.07 

Determines  the  number  of  target 
elements  that  are  destroyed  when  a 
given  weapons  area  of  effect  fully  or 
partially  covers  a  target. 

Detenrine  whether  a  live  target  element 
is  hit.  (U(0,l)<.prob.hit.live) 

AIR840_2 

-27766.20 

Determines  the  number  of  target 
elements  that  are  destroyed  when  a 
given  weapons  area  of  effect  fully  or 
partially  covers  a  target. 

Detenrine  whether  a  live  target  element 
is  hit.  (U(0,l)<.exp.live.hits) 

BSE4001 

-8446.20 

Detenrine  s  the  availability  of  aircraft, 
establishes  the  munition 
configurations,  allocates  fuels  and 
determines  whether  to  cancel  or  mn  the 

mission  as  scheduled. 

Detenrines  the  cancellation  of  a  sortie 
due  to  degrade. 

(U(0, 1  )<eawp .  s  ortie  .degrade) 

Using  the  chart  above,  a  number  of  system  characteristics  can  be  detennined. 

First,  by  increasing  in  the  saturation  delay  on  the  shooter,  there  will  be  a  small  increase  in 
BLUE  SMG.  Second,  it  can  be  determined  that  increasing  the  number  of  random  draws 
greater  than  the  engagement  probability  will  greatly  increase  the  BLUE  SMG.  Another 
way  to  look  at  this  is  decreasing  the  air-to-air  engagements  will  increase  the  BLUE  SMG. 
Third,  by  icreasing  the  success  of  a  mid-course  update  will  increase  the  BLUE  SMG. 
Fourth,  increasing  the  probability  of  hitting  a  live  target  will  greatly  increase  the  BLUE 
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SMG.  Fifth,  the  greatest  increase  in  BLUE  SMG  is  related  to  the  number  of  sorties.  If 
the  degrade  in  the  sorties  are  large,  then  the  number  of  sorties  are  reduced  and  the  BLUE 
SMG  is  also  reduced.  An  overall  look  at  this  implies  that  the  more  sorties  that  are 
generated  and  the  fewer  enemy  engagements  along  with  better  strikes  will  lead  to  a  more 
BLUE  Square  Miles  Gained.  This  same  analysis  can  be  applied  to  the  control  variates 
related  to  other  MOEs. 

4.1.4.  Replication  Speed  With  And  Without  Generating  Control  Variates. 

To  test  the  difference  in  running  times,  the  start  and  completion  times  for  each 
replication  is  recorded  and  the  difference  is  calculated  in  seconds.  Then  a  paired-t  Cl  is 
calculated  and  if  the  Cl  contains  zero,  it  is  detennined  that  the  times  are  not  statistically 
different.  The  300  replications  performed  while  producing  control  variate  infonnation, 
set  A,  are  compared  to  the  300  replications  perfonned  while  not  producing  control  variate 
information,  set  G,  by  subtracting  the  replication  times  of  set  G  from  set  A.  The 
confidence  interval  on  the  difference  is  (0.01284,  0.18049).  A  graphical  representation  is 
below. 


-0.1  -0.05  0  0.05  0.1  0.15  0.2 

Figure  4.3:  Replication  Completion  Difference  Confidence  Interval 


Since  the  confidence  interval  does  not  contain  zero,  the  completion  times  are  statistically 
different  at  the  a  =  0.05  level.  Since  the  times  for  replications  not  generating  the  control 


4.10 


variates  are  subtracted  from  the  times  for  replications  generating  the  control  variates,  the 
Cl  implies  that  the  times  for  generating  the  control  variates  are  larger. 

4.2.  Common  Random  Numbers 


To  analyze  the  effectiveness  of  the  CRN  technique,  a  total  of  1200  replications 
were  perfonned.  Table  4.8  details  the  experiment  settings;  remember  that  each  set 
consists  of  300  replications.  The  sets  can  be  separated  into  two  groups.  The  first  group 
is  the  standard  non-synchronized  data  that  makes  up  the  base  data  and  the  second  group 
is  the  synchronized  data.  All  replications  were  of  the  standard  ME  scenario,  using  the 
new  RNG.  The  first  300  replications,  set  B,  contained  no  changes  to  the  database  and 
with  all  random  inputs  using  a  single  stream  for  each  replication.  The  second  300 
replications,  set  C,  were  perfonned  with  all  of  the  F-l  1  Is  removed  from  the  database  and 
with  all  random  inputs  using  the  same  single  streams  applied  to  set  B.  The  third  300 
replications,  set  D,  contained  no  changes  to  the  database  and  were  performed  with  all  of 
the  random  inputs  synchronized  therefore,  receiving  their  own  random  number  stream  in 
each  replication.  The  last  300  replications,  set  E,  were  performed  with  all  of  the  F-l  1  Is 


removed  from  the  database  and  with  the  random  inputs  synchronized  with  set  D. 


Table  4.8:  Common  Random  Number  Experiment  Settings 


RNG 

SYNCHRONIZED 

ANTITHETIC 

COLLECT  CV 

CONFIGURATION 

SET 

(NEW/OLD) 

(YES/NO) 

(YES/NO) 

(YES/NO) 

(ORIGINAL/MODIFIED) 

B 

NEW 

NO 

NO 

NO 

ORIGINAL 

C 

NEW 

NO 

NO 

YES 

MODIFIED 

D 

NEW 

YES 

NO 

YES 

ORIGINAL 

E 

NEW 

YES 

NO 

YES 

MODIFIED 

The  following  analysis  compares  the  halfwidths  generated  by  calculating  the 


difference  between  scenario  configurations.  The  first  comparison  is  by  the  standard 
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method,  not  synchronized,  and  the  second  comparison  is  by  CRN  with  synchronization. 
The  second  section  highlights  insights  gleaned  from  using  CRN. 

4.2.1.  Normal  to  Variance  Reduction  Technique. 

The  paired-t  Cl  at  the  a  =  0.05  level  was  used  to  determine  whether  the  MOEs 


generated  from  a  configuration  change  in  the  ME  scenario  resulted  in  statistically 
different  values.  Then  the  paired-t  halfwidths  were  compared  to  determine  the  reduction 
in  variance.  The  following  table  displays  the  results  of  this  comparison. 


Table  4.9:  Common  Random  Number  Halfwidth  Reduction 


MOE 

BA 

MEAN 

SE 

HALFWIDTH 

CF 

MEAN 

IN 

HALFWIDTH 

PERCENT 

CHANGE 

A- 10  LPS 

-0.004978203 

0.00053875 

-0.005019134 

0.000533584 

-0.96% 

A-6  LPS 

-0.003968323 

0.001541612 

-0.003959031 

0.001382776 

-10.30% 

BLUE  FPS 

0.07850412 

0.00280076 

0.079916562 

0.002443302 

-12.76% 

BLUE  LPS 

-0.000834882 

0.000253594 

-0.00110882 

0.000234114 

-7.68% 

BLUE  SMG 

323.05 

109.3136609 

265.22 

115.3944755 

5.56% 

DHAHRAN  FPS 

0.047589307 

0.008795164 

0.046170201 

0.007928092 

-9.86% 

F- 15  LPS 

-0.000518151 

0.000438013 

-0.000617566 

0.000422892 

-3.45% 

FA- 18  FPS  total 

0.023195554 

0.004674708 

0.023448124 

0.004512142 

-3.48% 

FA- 18  FPS  DCA 

0.000753974 

0.001907906 

-0.000751392 

0.001799667 

-5.67% 

MIG-23  LPS 

0.010507245 

0.010701722 

0.016051307 

0.008963076 

-16.25% 

MIG-29  LPS 

0.052738368 

0.00954175 

0.0621385 

0.010117406 

6.03% 

MIRAGE  FPS  total 

0.01092006 

0.006458806 

0.008898713 

0.005904645 

-8.58% 

MIRAGE  FPS  INT 

-0.002057046 

0.007664981 

-0.002582203 

0.007259603 

-5.29% 

MIRAGE  LPS 

-0.001408885 

0.006220598 

0.005438013 

0.005672349 

-8.81% 

MUDAYSIS  FPS 

0.073151537 

0.005964334 

0.067373498 

0.005894866 

-1.16% 

RED  FPS 

0.034304208 

0.002341687 

0.032198334 

0.002392857 

2.19% 

RED  LPS 

0.02384662 

0.002443123 

0.023647381 

0.002339682 

-4.23% 

RIYADH  FPS 

-0.072238335 

0.001832508 

-0.071748036 

0.001917861 

4.66% 

SHAIBAH  FPS 

0.090617611 

0.009656621 

0.089769928 

0.010008527 

3.64% 

Looking  above,  we  can  see  that  the  halfwidth  is  not  always  smaller.  The  changes 


highlighted  above  are  the  halfwidths  that  are  reduced.  This  implies  that  there  is  not 
always  a  reduction  in  the  variance  when  using  CRN.  The  following  two  charts  are  a 


graphical  representation  of  the  largest  reduction,  MIG-23  LPS  and  the  largest  increase, 
MIG-29  LPS. 
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SYNCHRONIZED 


-0.02  -0.01 
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0  0.01  0.02 


0.03 


Figure  4.4:  MIG-23  Losses  per  Sortie  Value  Difference  Confidence  Intervals 

The  confidence  interval  generated  for  the  base  (-0.00019,0.021209)  and  for  the 
synchronized  (0.007088,0. 02501 4). 


-0.02  0  0.02  0.04  0.06  0.08 


Figure  4.5:  MIG-29  Losses  per  Sortie  Value  Difference  Confidence  Intervals 

The  confidence  interval  generated  for  the  base  (0.043197,0.06228)  and  for  the 
synchronized  (0.05202 1 ,0.072256). 


4.2.2.  Common  Random  Numbers  Insights. 

As  can  be  seen  above  in  Figure  4.4,  the  decision  of  whether  the  MOE  results 
between  configuration  are  statistically  different  or  not,  may  change  between  the  base 
non-synchronized  and  the  synchronized,  as  a  result  from  both  the  change  in  halfwidth  and 
a  shift  in  the  mean  value.  The  following  table  displays  the  decision  results  among  the 
MOEs  and  identifies  any  changes  in  the  decisions  between  the  base  and  synchronized 
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results.  Each  of  the  CIs  developed  for  the  MOEs  below  are  calculated  at  the  a  =  0.05 


level  and  are  calculated  separately. 


Table  4.10:  Common  Random  Number  Configuration  Decision 


MOE 

BASE 

DECISION 

CRN 

DECISION 

A-10LPS 

DIFFERENT 

DIFFERENT 

A-6  LPS 

DIFFERENT 

DIFFERENT 

BLUE  FPS 

DIFFERENT 

DIFFERENT 

BLUE  LPS 

DIFFERENT 

DIFFERENT 

BLUE  SMG 

DIFFERENT 

DIFFERENT 

DHAHRAN  FPS 

DIFFERENT 

DIFFERENT 

F- 15  LPS 

DIFFERENT 

DIFFERENT 

FA-18  FPS  total 

DIFFERENT 

DIFFERENT 

FA- 18  FPS  DCA 

NOT  DIFFERENT 

NOT  DIFFERENT 

MIG-23  LPS 

NOT  DIFFERENT 

DIFFERENT 

MIG-29  LPS 

DIFFERENT 

DIFFERENT 

MIRAGE  FPS  total 

DIFFERENT 

DIFFERENT 

MIRAGE  FPS  INT 

NOT  DIFFERENT 

NOT  DIFFERENT 

MIRAGE  LPS 

NOT  DIFFERENT 

NOT  DIFFERENT 

MUDAYSIS  FPS 

DIFFERENT 

DIFFERENT 

RED  FPS 

DIFFERENT 

DIFFERENT 

RED  LPS 

DIFFERENT 

DIFFERENT 

RIYADH  FPS 

DIFFERENT 

DIFFERENT 

SHAIBAH  FPS 

DIFFERENT 

DIFFERENT 

Looking  at  the  table  above  we  can  see  that  the  decision  changes  only  once,  and  it  is 


highlighted  in  gray. 

Since  the  change  is  not  consistently  positive  or  negative  and  the  MOEs  were 


selected  to  reflect  a  range  of  aggregation,  it  seems  logical  to  see  if  there  are  any 


commonalities  between  the  aggregation  levels  and  reduction.  When  sorted  into  the 


different  aggregation  levels  no  pattern  emerges.  The  only  item  of  possible  note  is  that  the 


data  collected  at  the  squadron  level  are  all  positive.  When  the  data  is  sorted  by  force, 
RED  and  BLUE,  there  is  also  no  notable  pattern. 
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4.3.  Antithetic  Variates 


A  total  of  600  replications  were  perfonned  to  analyze  the  affects  of  antithetic 
variates  on  THUNDER.  Table  4.1 1  details  the  experiment  settings;  remember  that  each 
set  consists  of  300  replications.  The  first  300  replications,  set  D,  were  used  to  generate 
base  confidence  intervals  for  comparison  and  the  first  half  of  the  antithetic  pairs.  The 
replications  were  performed  with  the  new  RNG,  with  each  random  input  synchronized 
with  its  own  stream,  and  antithetic  variates  not  selected.  The  second  300  replications,  set 
F,  were  the  second  half  of  the  antithetic  pairs.  The  replications  were  performed  with  the 
new  RNG,  with  each  random  number  input  synchronized  with  the  replications  from  set 
D,  and  antithetic  variates  turned  on.  Turning  on  antithetic  variates  forces  all  of  the 
random  numbers  to  be  drawn  as  1  -  the  original  random  number.  The  300  antithetic  pairs 
are  generated  from  all  600  replications.  The  halfwidths  were  calculated  at  the  a  =  0.05 
level. 


Table  4.11:  Antithetic  Variate  Experiment  Settings 


RNG 

SYNCHRONIZED 

ANTITHETIC 

COLLECT  CV 

CONFIGURATION 

SET 

(NEW/OLD) 

(YES/NO) 

(YES/NO) 

(YES/NO) 

(ORIGINAL/MODIFIED) 

D 

NEW 

YES 

NO 

YES 

ORIGINAL 

F 

NEW 

YES 

YES 

YES 

ORIGINAL 

The  first  subsection  contains  the  comparison  of  the  antithetic  variates  halfwidths 
with  the  regular  halfwidths.  The  second  subsection  contains  the  calculations  to  determine 
the  number  of  replications  needed  to  obtain  the  variance  reduction  obtained  with 
antithetic  variates.  The  last  subsection  highlights  insights  seen  in  antithetic  analysis. 
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4.3.1.  Normal  to  Variance  Reduction  Technique. 

To  test  the  reduction  in  variance,  the  standard  generated  halfwidths  of  the  MOEs 
are  compared  with  the  antithetic  variate  halfwidths  of  the  MOEs.  The  following  table 
reports  the  variance  reduction  due  to  antithetic  variates. 


Table  4.12:  Antithetic  Variate  (300  Replication)  vs.  Replication  Reduction 


MOE 

BA 

MEAN 

.SE 

HALFWIDTH 

ANTITHETIC 

MEAN 

;  VARIATES 

HALFWIDTH 

PERCENT 

CHANGE 

A-10LPS 

0.005145212 

0.00028569 

0.004980108 

0.000214215 

-25.02% 

A-6  LPS 

0.019104498 

0.000928243 

0.018614329 

0.000708626 

-23.66% 

BLUE  FPS 

0.582000237 

0.002036313 

0.581054499 

0.001481472 

-27.25% 

BLUE  LPS 

0.010170101 

0.000174156 

0.010111304 

0.000112451 

-35.43% 

BLUE  SMG 

34474.62 

84.52936072 

34526.50667 

57.67080159 

-31.77% 

DHAHRAN  FPS 

0.372019758 

0.006733921 

0.372211006 

0.004820585 

-28.41% 

F-111  LPS 

0.016760015 

0.000416308 

0.016685204 

0.000253913 

-39.01% 

F- 15  LPS 

0.006207278 

0.00031009 

0.006242106 

0.000208184 

-32.86% 

FA- 18  FPS  TOTAL 

0.233627904 

0.003723047 

0.232053597 

0.002729652 

-26.68% 

FA- 18  FPS  DCA 

0.044337541 

0.001389283 

0.043733405 

0.000830637 

-40.21% 

MIG-23  LPS 

0.257329967 

0.006037934 

0.257671361 

0.004167409 

-30.98% 

MIG-29  LPS 

0.200408126 

0.007942862 

0.197127827 

0.006042087 

-23.93% 

MIRAGE  FPS  TOTAL 

0.600422202 

0.00449042 

0.601209538 

0.003406191 

-24.15% 

MIRAGE  FPS  INT 

0.699855667 

0.005731393 

0.702536288 

0.003813333 

-33.47% 

MIRAGE  LPS 

0.276394858 

0.004501578 

0.275381579 

0.003051761 

-32.21% 

MUDAYSIS  FPS 

0.189549709 

0.00488572 

0.1887675 

0.003383585 

-30.75% 

RED  FPS 

0.391872732 

0.001772589 

0.392056037 

0.001094404 

-38.26% 

RED  LPS 

0.192619582 

0.001692612 

0.191275098 

0.001079866 

-36.20% 

RIYADH  FPS 

0.854947018 

0.001535138 

0.854861142 

0.00111826 

-27.16% 

SHAIBAH  FPS 

0.504592164 

0.006312332 

0.50172746 

0.004368135 

-30.80% 

The  reduction  in  halfwidths  in  the  above  table  initially  looks  great,  but  the  fact  that  the 


antithetic  variates  are  generated  with  twice  the  number  of  replications  may  inflate  the 
actual  reduction.  Therefore,  the  following  antithetic  variate  vs.  replication  table  was 


created  using  only  the  first  150  antithetic  pairs. 
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Table  4.13:  Antithetic  Variate  (150  Replication)  vs.  Replication  Reduction 


MOE 

BA 

MEAN 

lSE 

HALFWIDTH 

ANTITHETIC 

MEAN 

;  VARIATES 

HALFWIDTH 

PERCENT 

CHANGE 

A-10LPS 

0.005145212 

0.00028569 

0.004980108 

0.000304497 

6.58% 

A-6  LPS 

0.019104498 

0.000928243 

0.018614329 

0.001007279 

8.51% 

BLUE  FPS 

0.582000237 

0.002036313 

0.581054499 

0.002105844 

3.41% 

BLUE  LPS 

0.010170101 

0.000174156 

0.010111304 

0.000159844 

-8.22% 

BLUE  SMG 

34474.62 

84.52936072 

34526.50667 

81 .9763507 

-3.02% 

DHAHRAN  FPS 

0.372019758 

0.006733921 

0.372211006 

0.006852237 

1.76% 

F-111  LPS 

0.016760015 

0.000416308 

0.016685204 

0.000360925 

-13.30% 

F- 15  LPS 

0.006207278 

0.00031009 

0.006242106 

0.000295924 

-4.57% 

FA- 18  FPS  TOTAL 

0.233627904 

0.003723047 

0.232053597 

0.003880073 

4.22% 

FA- 18  FPS  DCA 

0.044337541 

0.001389283 

0.043733405 

0.001180712 

-15.01% 

MIG-23  LPS 

0.257329967 

0.006037934 

0.257671361 

0.005923777 

-1.89% 

MIG-29  LPS 

0.200408126 

0.007942862 

0.197127827 

0.008588544 

8.13% 

MIRAGE  FPS  TOTAL 

0.600422202 

0.00449042 

0.601209538 

0.004841742 

7.82% 

MIRAGE  FPS  INT 

0.699855667 

0.005731393 

0.702536288 

0.005420474 

-5.42% 

MIRAGE  LPS 

0.276394858 

0.004501578 

0.275381579 

0.004337935 

-3.64% 

MUDAYSIS  FPS 

0.189549709 

0.00488572 

0.1887675 

0.004809609 

-1.56% 

RED  FPS 

0.391872732 

0.001772589 

0.392056037 

0.001555645 

-12.24% 

RED  LPS 

0.192619582 

0.001692612 

0.191275098 

0.001534979 

-9.31% 

RIYADH  FPS 

0.854947018 

0.001535138 

0.854861142 

0.001589555 

3.54% 

SHAIBAH  FPS 

0.504592164 

0.006312332 

0.50172746 

0.006209099 

-1.64% 

Now,  with  the  number  of  replications  used  to  generate  the  base  and  antithetic  pairs  being 


equal,  the  reduction  becomes  mixed.  The  halfwidth  reductions  are  highlighted  above. 


4.3.2.  Variance  Reduction  Technique  to  Number  of  Replications. 

The  reduction  gained  can  also  be  seen  by  calculating  the  number  of  replications 
needed  to  achieve  the  antithetic  variate  halfwidth.  To  do  this,  the  halfwidth  calculation  is 


iterated  by  incrementing  the  number  of  replications  in  the  equation  until  the  halfwidth  is 
smaller  than  the  antithetic  variate  halfwidth.  A  major  assumption  to  this  method  is  the 
variance  remains  constant.  The  following  table  displays  the  number  of  replications 
needed  to  achieve  the  variance  reduction  with  the  300  antithetic  pairs. 


4.17 


Table  4.14:  Replications  Needed  to  Achieve  Antithetic  Variate  Reduction  (300 

Replications) 


MOE 

REPLICATIONS 

CHANGE  IN 

REPLICATIONS 

PERCENT 

CHANGE 

A-10LPS 

532 

232 

77.33% 

A-6  LPS 

513 

213 

71.00% 

BLUE  FPS 

565 

265 

88.33% 

BLUE  LPS 

716 

416 

138.67% 

BLUE  SMG 

642 

342 

114.00% 

DHAHRAN  FPS 

583 

283 

94.33% 

F-111  LPS 

802 

502 

167.33% 

F- 15  LPS 

662 

362 

120.67% 

FA-18  FPS  total 

556 

256 

85.33% 

FA- 18  FPS  DCA 

834 

534 

178.00% 

MIG-23  LPS 

627 

327 

109.00% 

MIG-29  LPS 

517 

217 

72.33% 

MIRAGE  FPS  total 

520 

220 

73.33% 

MIRAGE  FPS  INT 

674 

374 

124.67% 

MIRAGE  LPS 

650 

350 

116.67% 

MUDAYSIS  FPS 

623 

323 

107.67% 

RED  FPS 

783 

483 

161.00% 

RED  LPS 

733 

433 

144.33% 

RIYADH  FPS 

563 

263 

87.67% 

SHAIBAH  FPS 

624 

324 

108.00% 

Note  above,  that  not  all  of  the  replication  increases  are  greater  than  or  equal  to  the  300 
extra  replications  needed  to  create  the  antithetic  pairs.  The  following  table  is  a  better 
representation  of  the  number  of  the  number  of  replications  needed  to  achieve  the  variance 


reduction.  It  uses  the  antithetic  variates  with  the  150  replications. 
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Table  4.15:  Replications  Needed  to  Achieve  Antithetic  Variate  Reduction  (150 

Replications) 


MOE 

REPLICATIONS 

CHANGE  IN 

REPLICATIONS 

PERCENT 

CHANGE 

A-10LPS 

265 

-35 

-1 1 .67% 

A-6  LPS 

256 

-44 

-14.67% 

BLUE  FPS 

281 

-19 

-6.33% 

BLUE  LPS 

356 

56 

18.67% 

BLUE  SMG 

319 

19 

6.33% 

DHAHRAN  FPS 

290 

-10 

-3.33% 

F-111  LPS 

399 

99 

33.00% 

F- 15  LPS 

330 

30 

10.00% 

FA-18  FPS  total 

277 

-23 

-7.67% 

FA- 18  FPS  DCA 

415 

115 

38.33% 

MIG-23  LPS 

312 

12 

4.00% 

MIG-29  LPS 

258 

-42 

-14.00% 

MIRAGE  FPS  total 

259 

-41 

-13.67% 

MIRAGE  FPS  INT 

336 

36 

12.00% 

MIRAGE  LPS 

323 

23 

7.67% 

MUDAYSIS  FPS 

310 

10 

3.33% 

RED  FPS 

389 

89 

29.67% 

RED  LPS 

365 

65 

21.67% 

RIYADH  FPS 

281 

-19 

-6.33% 

SHAIBAH  FPS 

310 

10 

3.33% 

As  seen  in  the  previous  section,  not  all  halfwidths  are  reduced.  Also,  note  that  the  MOEs 


in  Table  4. 14,  that  have  an  increase  less  than  300,  are  the  same  MOEs  in  the  table  above 


that  have  increases. 

4.3.3.  Antithetic  Variates  Insights. 

As  with  common  random  numbers,  since  the  reduction  is  not  always  positive,  a 
search  for  commonalities  between  the  aggregation  levels  and  reduction  could  be  fruitful. 
When  sorted  into  the  different  aggregation  levels  no  pattern  emerges.  Also,  when  sorted 
by  force,  RED  and  BLUE,  no  pattern  emerges. 

4.4.  Control  Variates  and  Antithetic  Variates 


The  same  600  replications  used  for  antithetic  variates  were  also  used  for  the 
combination  of  control  variates  and  antithetic  variates.  Table  4.16  details  the  experiment 
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settings;  remember  that  each  set  consists  of  300  replications.  The  first  300  replications, 
set  D,  were  used  to  generate  base  confidence  intervals  for  comparison  and  the  first  half  of 
the  antithetic  pairs.  The  replications  were  perfonned  with  the  new  RNG,  with  each 
random  input  synchronized  with  its  own  stream,  and  antithetic  variates  not  selected.  The 
second  300  replications,  set  F,  were  the  second  half  of  the  antithetic  pairs.  The 
replications  were  performed  with  the  new  RNG,  with  each  random  number  input 
synchronized  with  the  replications  from  set  D,  and  antithetic  variates  turned  on.  Turning 
on  antithetic  variates  forces  all  of  the  random  numbers  to  be  drawn  as  1  -  the  original 
random  number.  The  only  difference  is  that  control  variates  are  used  to  reduce  the 


variance.  The  halfwidths  were  calculated  with  at  the  a  =  0.05  level. 


Table  4.16:  Antithetic  Variate  /  Control  Variate  Experiment  Settings 


RNG 

SYNCHRONIZED 

ANTITHETIC 

COLLECT  CV 

CONFIGURATION 

SET 

(NEW/OLD) 

(YES/NO) 

(YES/NO) 

(YES/NO) 

(ORIGINAL/MODIFIED) 

D 

NEW 

YES 

NO 

YES 

ORIGINAL 

F 

NEW 

YES 

YES 

YES 

ORIGINAL 

The  first  subsection  contains  the  comparison  of  the  base  halfwidths  to  the  control 
variate  and  antithetic  variate  reduction.  The  second  subsection  contains  an  analysis  of  the 
number  of  replications  needed  to  achieve  the  control  variate  and  antithetic  variate 
reduction.  The  last  subsection  contains  insights  obtained  by  using  antithetic  variates  in 
combination  with  control  variates. 

4.4.1.  Normal  to  Variance  Reduction  Technique. 

To  test  the  reduction  in  variance,  the  standard  generated  halfwidths  of  the  MOEs 
are  compared  with  the  antithetic  variate  halfwidths  of  the  MOEs  reduced  even  further  by 
also  applying  the  control  variate  technique.  The  following  table  reports  the  variance 
reduction  due  to  antithetic  variates  and  control  variates. 
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Table  4.17:  Control  Variate  and  Antithetic  Variate  (300  Replication)  vs.  Replication 

Reduction 


MOE 

BA 

MEAN 

iSE 

HALFWIDTH 

A Vj 

MEAN 

'CV 

HALFWIDTH 

PERCENT 

CHANGE 

A-10LPS 

0.005145212 

0.00028569 

0.004999208 

0.000195998 

-31 .40% 

A-6  LPS 

0.019104498 

0.000928243 

0.01877966 

0.000687489 

-25.94% 

BLUE  FPS 

0.582000237 

0.002036313 

0.580978958 

0.001410833 

-30.72% 

BLUE  LPS 

0.010170101 

0.000174156 

0.010142479 

0.000115121 

-33.90% 

BLUE  SMG 

34474.62 

84.52936072 

34473.68258 

61.43489655 

-27.32% 

DHAHRAN  FPS 

0.372019758 

0.006733921 

0.369991922 

0.004627463 

-31 .28% 

F-111  LPS 

0.016760015 

0.000416308 

0.016805487 

0.000266501 

-35.98% 

F- 15  LPS 

0.006207278 

0.00031009 

0.006329536 

0.00020287 

-34.58% 

FA- 18  FPS  TOTAL 

0.233627904 

0.003723047 

0.232572805 

0.002590169 

-30.43% 

FA- 18  FPS  DCA 

0.044337541 

0.001389283 

0.044025386 

0.000931742 

-32.93% 

MIG-23  LPS 

0.257329967 

0.006037934 

0.256304079 

0.004466771 

-26.02% 

MIG-29  LPS 

0.200408126 

0.007942862 

0.198779022 

0.005634799 

-29.06% 

MIRAGE  FPS  TOTAL 

0.600422202 

0.00449042 

0.600109645 

0.003195737 

-28.83% 

MIRAGE  FPS  INT 

0.699855667 

0.005731393 

0.700002069 

0.003816317 

-33.41% 

MIRAGE  LPS 

0.276394858 

0.004501578 

0.276342171 

0.002949469 

-34.48% 

MUDAYSIS  FPS 

0.189549709 

0.00488572 

0.189966945 

0.003417543 

-30.05% 

RED  FPS 

0.391872732 

0.001772589 

0.391871895 

0.001191619 

-32.78% 

RED  LPS 

0.192619582 

0.001692612 

0.192330512 

0.001167319 

-31 .03% 

RIYADH  FPS 

0.854947018 

0.001535138 

0.854828925 

0.001071476 

-30.20% 

SHAIBAH  FPS 

0.504592164 

0.006312332 

0.505066145 

0.004503055 

-28.66% 

Again,  this  reduction  is  created  with  a  total  of  600  replications,  double  the  number  of 
replications  used  for  the  base  case.  The  next  table  is  the  reduction  due  to  antithetic 
variates  using  only  the  first  150  antithetic  pairs  and  control  variates. 
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Table  4.18:  Control  Variate  and  Antithetic  Variate  (150  Replication)  vs.  Replication 

Reduction 


MOE 

BA 

MEAN 

iSE 

HALFWIDTH 

AVi 

MEAN 

'CV 

HALFWIDTH 

PERCENT 

CHANGE 

A-10LPS 

0.005145212 

0.00028569 

0.005038231 

0.000292638 

2.43% 

A-6  LPS 

0.019104498 

0.000928243 

0.018696752 

0.00100282 

8.03% 

BLUE  FPS 

0.582000237 

0.002036313 

0.581084169 

0.002088085 

2.54% 

BLUE  LPS 

0.010170101 

0.000174156 

0.010110917 

0.000156743 

-10.00% 

BLUE  SMG 

34474.62 

84.52936072 

34547.7152 

80.62975389 

-4.61% 

DHAHRAN  FPS 

0.372019758 

0.006733921 

0.370927039 

0.006942557 

3.10% 

F-111  LPS 

0.016760015 

0.000416308 

0.016744672 

0.00034699 

-16.65% 

F- 15  LPS 

0.006207278 

0.00031009 

0.006303071 

0.000289469 

-6.65% 

FA- 18  FPS  TOTAL 

0.233627904 

0.003723047 

0.231209234 

0.003892977 

4.56% 

FA- 18  FPS  DCA 

0.044337541 

0.001389283 

0.043528432 

0.001156129 

-16.78% 

MIG-23  LPS 

0.257329967 

0.006037934 

0.257478208 

0.005841578 

-3.25% 

MIG-29  LPS 

0.200408126 

0.007942862 

0.194782302 

0.007954705 

0.15% 

MIRAGE  FPS  TOTAL 

0.600422202 

0.00449042 

0.600955714 

0.004651799 

3.59% 

MIRAGE  FPS  INT 

0.699855667 

0.005731393 

0.702804931 

0.005358865 

-6.50% 

MIRAGE  LPS 

0.276394858 

0.004501578 

0.27581175 

0.004245828 

-5.68% 

MUDAYSIS  FPS 

0.189549709 

0.00488572 

0.189203986 

0.00448441 

-8.21% 

RED  FPS 

0.391872732 

0.001772589 

0.392220779 

0.001535464 

-13.38% 

RED  LPS 

0.192619582 

0.001692612 

0.191270973 

0.001510308 

-10.77% 

RIYADH  FPS 

0.854947018 

0.001535138 

0.854997708 

0.001574458 

2.56% 

SHAIBAH  FPS 

0.504592164 

0.006312332 

0.501744195 

0.006108584 

-3.23% 

Again,  the  reduction  is  mixed  and  the  reduced  halfwidths  are  highlighted.  When 


compared  to  the  regular  antithetic  variates,  the  halfwidth  reduction  is  larger  and  the 
halfwidth  increase  is  smaller. 


4.4.2.  Variance  Reduction  Technique  to  Number  of  Replications. 

Anther  way  to  represent  the  reduction  realized  from  the  combination  of  control 
variates  and  antithetic  variates  is  seen  by  calculating  the  number  of  replications  needed  to 
achieve  the  control  variate  and  antithetic  variate  halfwidth.  To  do  this,  the  halfwidth 
calculation  is  iterated  by  incrementing  the  number  of  replications  in  the  equation  until  the 
halfwidth  is  smaller  than  the  antithetic  variate  halfwidth.  A  major  assumption  to  this 
method  is  the  variance  remains  constant.  The  following  table  displays  the  number  of 
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replications  needed  to  achieve  the  variance  reduction  created  by  the  300  replication 


antithetic  variates  and  control  variates. 

Table  4.19:  Replications  Needed  to  Achieve  Control  Variate  and  Antithetic  Variate 

Reduction  (300  replications) 


MOE 

REPLICATIONS 

CHANGE  IN 

REPLICATIONS 

PERCENT 

CHANGE 

A-10LPS 

635 

335 

1 1 1 .67% 

A-6  LPS 

545 

245 

81.67% 

BLUE  FPS 

622 

322 

107.33% 

BLUE  LPS 

683 

383 

127.67% 

BLUE  SMG 

566 

266 

88.67% 

DHAHRAN  FPS 

632 

332 

110.67% 

F-111  LPS 

728 

428 

142.67% 

F- 15  LPS 

697 

397 

132.33% 

FA-18  FPS  total 

617 

317 

105.67% 

FA- 18  FPS  DCA 

664 

364 

121.33% 

MIG-23  LPS 

546 

246 

82.00% 

MIG-29  LPS 

594 

294 

98.00% 

MIRAGE  FPS  total 

590 

290 

96.67% 

MIRAGE  FPS  INT 

673 

373 

124.33% 

MIRAGE  LPS 

695 

395 

131.67% 

MUDAYSIS  FPS 

610 

310 

103.33% 

RED  FPS 

661 

361 

120.33% 

RED  LPS 

628 

328 

109.33% 

RIYADH  FPS 

613 

313 

104.33% 

SHAIBAH  FPS 

587 

287 

95.67% 

Note  again,  that  not  all  of  the  replication  increases  are  greater  than  or  equal  to  the  300 
extra  replications  needed  to  create  the  antithetic  variates.  The  following  table  displays 
the  number  of  replications  needed  to  achieve  the  halfwidth  generated  by  the  combination 
of  control  variates  and  antithetic  variates  with  the  150  antithetic  pairs. 
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Table  4.20:  Replications  Needed  to  Achieve  Control  Variate  and  Antithetic  Variate 

Reduction  (150  replications) 


MOE 

REPLICATIONS 

CHANGE  IN 

REPLICATIONS 

PERCENT 

CHANGE 

A-10LPS 

287 

-13 

-4.33% 

A-6  LPS 

258 

-42 

-14.00% 

BLUE  FPS 

286 

-14 

-4.67% 

BLUE  LPS 

370 

70 

23.33% 

BLUE  SMG 

330 

30 

10.00% 

DHAHRAN  FPS 

283 

-17 

-5.67% 

F-111  LPS 

431 

131 

43.67% 

F- 15  LPS 

344 

44 

14.67% 

FA-18  FPS  total 

275 

-25 

-8.33% 

FA- 18  FPS  DCA 

432 

132 

44.00% 

MIG-23  LPS 

321 

21 

7.00% 

MIG-29  LPS 

300 

0 

0.00% 

MIRAGE  FPS  total 

280 

-20 

-6.67% 

MIRAGE  FPS  INT 

343 

43 

14.33% 

MIRAGE  LPS 

337 

37 

12.33% 

MUDAYSIS  FPS 

356 

56 

18.67% 

RED  FPS 

399 

99 

33.00% 

RED  LPS 

377 

77 

25.67% 

RIYADH  FPS 

286 

-14 

-4.67% 

SHAIBAH  FPS 

321 

21 

7.00% 

4.4.3.  Combined  Antithetic  Variate  and  Control  Variate  Insight. 

Since  the  control  variate  technique  is  applied  to  the  antithetic  variates, 
information  can  be  gleaned  as  in  section  4. 1 .3  Control  Variate  Insight.  A  list  of 
statistically  significant  controls  related  to  the  MOEs  from  the  150-replication  antithetic 
pairs  is  below. 
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Table  4.21:  Antithetic  Variate  and  Control  Variate  Statistically  Significant  Controls 

Table  (150  Replications) 


MOE 

WEIGHT 

CONTROL 

MOE 

WEIGHT 

CONTROL 

A-10LPS 

-0.0077 

MIG-29  LPS 

15.1554 

ISsSEESil 

0.0082 

-1 .2270 

0.2273 

33.4814 

A-6  LPS 

2.8138 

-4.4824 

BLUE  FPS 

-0.0542 

-36.7484 

BLUE  LPS 

0.3643 

MIRAGE  FPS 

TOTAL 

0.0583 

BLUE  SMG 

33390.9611 

0.0006 

-112989.0889 

-7.0484 

DHAHRAN  FPS 

17.9111 

MIRAGE  FPS  INT 

0.8158 

F-111  LPS 

-0.2916 

MIRAGE  LPS 

-13.9508 

BBEB 

1.5291 

6.8540 

0.9901 

■ddrtliliM 

MUDAYSIS  FPS 

-10.8821 

F- 15  LPS 

0.0968 

10.0131 

0.2306 

■SMsUM 

0.0405 

0.3501 

0.0007 

naa i 

FA- 18  FPS 

TOTAL 

11.4841 

-14.8256 

AIR840 2 

1 .2846 

MlrfM— 

24.6841 

FA- 18  FPS  DCA 

-0.0034 

AIR060  2 

RED  FPS 

0.9477 

0.0146 

raua:i0jgaB 

0.6997 

ma 

MIG-23  LPS 

0.0637 

RED  LPS 

-0.8878 

RIYADH  FPS 

0.9135 

UMiaainngM 

SHAIBAH  FPS 

3.6016 

ma 

As  seen  with  the  CV,  these  weights  and  controls  can  be  used  to  give  the  analyst  a  better 


understanding  of  how  the  simulation  works.  The  following  table.  Table  4.22,  examines 


the  MOE  MUDAYSIS  FPS  (Flown  per  Planned  Sortie).  It  can  be  seen  that  the 
statistically  significant  controls  are  ADF140.3,  AIR007.1,  AIR070.1,  AIR567.2, 
AIR840.2,  and  BSE200.5.  Note,  AIR007. 1  is  a  random  input  that  is  miss  labeled.  It 
should  be  in  AIR007.sim,  but  it  is  located  in  AIR014.1. 
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Table  4.22:  Antithetic  Variate  and  Control  Variate  Purpose  of  Statistically 

Significant  Controls 


CONTROL 

WEIGHT 

MODULE  PURPOSE 

SPECIFIC  DRAW  PURPOSE 

ADF1403 

-10.8821 

Calculates,  for  each  air  defense  type,  the 
number  of  engaged  sites  and  rounds  fired 
per  site  for  primaries. 

Used  determine  the  actual  number  of  sites 
engaging  as  a  function  of 
FRAC.INTEGRATED  (used  to  measure 
command  and  control).  (U(0,1)< 

If  ac.  f( .  engaged,  shooters)) 

AIR007  1 

10.0131 

Continue  a  flight  after  the  air-to-air 
refueling  decision  has  been  made. 

Used  to  determine  the  launching  of 
interceptors. 

(U(0,  l)<=TA.PROB.ENEMY.INT.LAUN 
CH) 

AIR070  1 

0.0405 

Adjudicate  a  defensive  anti  tactical  ballistic 

missile  mission  VS.  tactical  ballistic  missile 
(TBM)  air-to-air  engagement. 

Used  to  determine  determine  TBM  flight 
phase  (and  pK)  at  which  to  adjudicate, 
(distribution)  Used  to  determine  duration 
of  aiborne  laser  shot  duration. 

AIR567  2 

0.0007 

Compute  target  location  error  for  an 
offensive  anti  tactical  ballistic  missile 
mission  (OTBM)  flight  group  which 
has  arrived  at  its  target. 

Generate  meters  error  from  a  normal 
distribution,  (distribution) 

A1R840_2 

-14.8256 

Determine  the  number  of  target  elements 
that  are  destroyed  when  a  given  weapons 
area  of  effect  hilly  or  partially  covers  a 
target. 

Determine  how  many  live  target  elements 
were  hit.  (U(0,1)<.PROB.HIT.LIVE) 

BSE2005 

24.6841 

Determine  maintenance  tasks  to  be 
performed  on  aircraft  in  the  given  flight 
and  schedule  those  for  which  resources  are 
currently  available.  Print  warning  when 
required  non-consumable  resource  is  not 
available  at  squadron's  mob. 

Used  to  build  list  of  maintenance  tasks 
required. 

(U(0,  l)<=TA.PROB.LONG.TERM.REPA 
IR 

Using  the  chart  above,  a  number  of  system  characteristics  can  be  detennined. 
First,  an  increase  in  the  number  of  engaged  air  defense  shooters  would  decrease 
MUDAYSIS  FPS.  Second,  a  decrease  in  the  launching  of  interceptors  would  decrease 
MUDAYSIS  FPS.  Third,  a  decrease  in  the  airborne  laser  shot  duration  required  will 
slightly  decrease  MUDAYSIS  FPS.  Fourth,  a  decrease  in  the  nonnal  error  of  an  OTBM 
(Offensive  Anti  Tactical  Ballistic  Missile  Mission)  will  slightly  decrese  MUDAYSIS 
FPS.  Fifth,  an  increse  in  the  number  of  live  targets  hit  will  decrease  MUDAYSIS  FPS. 
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Sixth,  the  increase  in  the  long-tenn  repairs,  will  greatly  decrese  MUDAYSIS  FPS. 
Overall,  it  seems  that  an  increse  in  engagement  will  decrese  the  number  of  flown  per 
planned  sorties.  This  same  analysis  can  be  applied  to  the  control  variates  related  to  other 
MOEs. 

As  with  common  random  numbers,  since  the  reduction  is  not  always  positive,  a 
search  for  commonalities  between  the  aggregation  levels  and  reduction  could  be  fruitful. 
When  sorted  into  the  different  aggregation  levels  no  pattern  emerges.  Also,  when  sorted 
by  force,  no  pattern  emerges. 

4.5.  Control  Variates  and  Common  Random  Numbers 

To  analyze  the  effectiveness  of  combining  CV  and  CRN  techniques,  a  total  of 
1200  replications  were  perfonned.  Table  4.8  details  the  experiment  settings;  remember 
that  each  set  consists  of  300  replications.  The  sets  can  be  separated  into  two  groups.  The 
first  group  is  the  standard  independent  data  that  makes  up  the  base  data  and  the  second 
group  is  the  partially  synchronized  data.  All  replications  were  of  the  standard  ME 
scenario,  using  the  new  RNG.  The  first  300  replications,  set  M,  contained  no  changes  to 
the  database  and  with  all  random  inputs  using  a  single  stream  for  each  replication.  The 
second  300  replications,  set  O,  were  perfonned  with  all  of  the  F-l  1  Is  removed  from  the 
database  and  with  all  random  inputs  using  the  same  single  streams  applied  to  set  B.  The 
third  300  replications,  set  S,  contained  no  changes  to  the  database  and  were  performed 
with  selected  random  inputs  synchronized  therefore,  receiving  their  own  random  number 
stream  in  each  replication  and  the  rest  independent.  The  last  300  replications,  set  T,  were 
performed  with  all  of  the  F-l  1  Is  removed  from  the  database  and  with  the  selected 
random  inputs  synchronized  with  set  S  and  the  rest  independent. 
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Table  4.23:  Control  Variate  and  Common  Random  Number  Experiment  Settings 


RNG 

SYNCHRONIZED 

ANTITHETIC 

COLLECT  CV 

CONFIGURATION 

SET 

(NEW/OLD) 

(YES/NO) 

(YES/NO) 

(YES/NO) 

(ORIGINAL/MODIFIED) 

NEW 

NO* 

NO 

YES 

ORIGINAL 

0 

NEW 

NO* 

NO 

YES 

MODIFIED 

s 

NEW 

YES* 

NO 

YES 

ORIGINAL 

T 

NEW 

YES* 

NO 

YES 

MODIFIED 

The  following  analysis  compares  the  halfwidths  generated  by  calculating  the 


difference  between  scenario  configurations.  The  first  comparison  is  by  the  standard 


method,  not  synchronized,  and  the  second  comparison  is  by  combining  CRN  with 


selected  synchronization  and  CV.  The  second  section  highlights  insights  gleaned  from 


using  CRN. 


4.5.1.  Normal  to  Variance  Reduction  Technique. 


The  paired-t  Cl  at  the  a  =  0.05  level  was  used  to  determine  whether  the  MOEs 


generated  from  a  configuration  change  in  the  ME  scenario  resulted  in  statistically 
different  values.  Then  the  paired-t  halfwidths  were  compared  to  determine  the  reduction 
in  variance.  The  following  table  displays  the  results  of  this  comparison. 
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Table  4.24:  Control  Variate  and  Common  Random  Number  Halfwidth  Reduction 


MOE 

BA 

MEAN 

SE 

HALFWIDTH 

cv/c 

MEAN 

;rn 

HALFWIDTH 

PERCENT 

CHANGE 

A- 10  LPS 

-0.005074435 

0.000471166 

-0.005420376 

0.000495875 

5.24% 

A-6  LPS 

-0.003709347 

0.001243332 

-0.003862475 

0.001142089 

-8.14% 

BLUE  FPS 

0.080204272 

0.002488093 

0.078021187 

0.002276083 

-8.52% 

BLUE  LPS 

-0.000980203 

0.000224242 

-0.001228896 

0.000216219 

-3.58% 

BLUE  SMG 

321.6566667 

92.75065974 

283.8370489 

94.46705704 

1.85% 

DHAHRAN  FPS 

0.047497651 

0.00785785 

0.043664487 

0.007499356 

-4.56% 

F-15  LPS 

-0.00029301 

0.000348631 

-0.000778342 

0.00039424 

13.08% 

FA-18  FPS  total 

0.023885856 

0.004348857 

0.021403974 

0.004115626 

-5.36% 

FA-18  FPS  DCA 

9.3761 6E-05 

0.001776789 

-0.002378309 

0.001642749 

-7.54% 

MIG-23  LPS 

0.00720557 

0.009116531 

0.002253995 

0.008420355 

-7.64% 

MIG-29  LPS 

0.058508654 

0.008627634 

0.058072515 

0.008322543 

-3.54% 

MIRAGE  FPS  total 

0.008655135 

0.005768579 

0.001181363 

0.00581596 

0.82% 

MIRAGE  FPS  INT 

-0.004831629 

0.006898662 

-0.001808304 

0.007049026 

2.18% 

MIRAGE  LPS 

-0.000117038 

0.00564862 

0.005595717 

0.005615011 

-0.60% 

MUDAYSIS  FPS 

0.071070466 

0.00554528 

0.066758129 

0.005276546 

-4.85% 

RED  FPS 

0.034613556 

0.002097873 

0.030483796 

0.002043576 

-2.59% 

RED  LPS 

0.022702029 

0.002120665 

0.02316245 

0.002015068 

-4.98% 

RIYADH  FPS 

-0.072184072 

0.001689959 

-0.072668778 

0.001737694 

2.82% 

SHAIBAH  FPS 

0.093599432 

0.008845515 

0.095513973 

0.008256205 

-6.66% 

Looking  above,  we  can  see  that  the  halfwidth  is  not  always  smaller.  The  changes 


highlighted  above  are  the  halfwidths  that  are  reduced.  This  implies  that  there  is  not 


always  a  reduction  in  the  variance  when  combining  CV  and  CRN.  Knowing  from  section 


4.2,  that  CRN  does  not  always  reduce  the  halfwidth,  the  control  variate  estimator  is 


applied  to  the  independent  base  data  set  to  detennine  the  effect  due  to  CV.  The  following 


table  is  the  result. 
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Table  4.25:  Control  Variate  Only  Halfwidth  Reduction 


MOE 

BA 

MEAN 

SE 

HALFWIDTH 

C 

MEAN 

V 

HALFWIDTH 

PERCENT 

CHANGE 

A-10LPS 

-0.005074435 

0.000471166 

-0.00506369 

0.000464018 

-1.52% 

A-6  LPS 

-0.003709347 

0.001243332 

-0.003673866 

0.001221513 

-1.75% 

BLUE  FPS 

0.080204272 

0.002488093 

0.080081841 

0.002427363 

-2.44% 

BLUE  LPS 

-0.000980203 

0.000224242 

-0.000971024 

0.000219306 

-2.20% 

BLUE  SMG 

321.6566667 

92.75065974 

322.6088152 

92.30977839 

-0.48% 

DHAHRAN  FPS 

0.047497651 

0.00785785 

0.047189217 

0.007795751 

-0.79% 

F- 15  LPS 

-0.00029301 

0.000348631 

-0.000292464 

0.000345254 

-0.97% 

FA- 18  FPS  total 

0.023885856 

0.004348857 

0.023798247 

0.004325066 

-0.55% 

FA- 18  FPS  DCA 

9.3761 6E-05 

0.001776789 

0.000161301 

0.001764302 

-0.70% 

MIG-23  LPS 

0.00720557 

0.009116531 

0.006046667 

0.008687175 

-4.71% 

MIG-29  LPS 

0.058508654 

0.008627634 

0.058064528 

0.008499025 

-1 .49% 

MIRAGE  FPS  total 

0.008655135 

0.005768579 

0.009424086 

0.005625573 

-2.48% 

MIRAGE  FPS  INT 

-0.004831629 

0.006898662 

-0.005030783 

0.006825481 

-1 .06% 

MIRAGE  LPS 

-0.000117038 

0.00564862 

-0.000991526 

0.005563242 

-1.51% 

MUDAYSIS  FPS 

0.071070466 

0.00554528 

0.071135554 

0.005548611 

0.06% 

RED  FPS 

0.034613556 

0.002097873 

0.034764827 

0.002032008 

-3.14% 

RED  LPS 

0.022702029 

0.002120665 

0.02226502 

0.002083652 

-1.75% 

RIYADH  FPS 

-0.072184072 

0.001689959 

-0.072522867 

0.001643588 

-2.74% 

SHAIBAH  FPS 

0.093599432 

0.008845515 

0.096093296 

0.008691198 

-1.74% 

Note,  that  almost  all  of  the  halfwidths  are  reduced.  This  implies  that  the  majority 


variance  increases  are  due  to  the  application  of  CRN. 


4.5.2.  Combined  Control  Variate  and  Common  Random  Number  Insight. 


As  can  be  seen  with  CRN,  the  decision  of  whether  the  MOE  results  between 
configurations  are  statistically  different  or  not,  may  change  between  the  base  non- 
synchronized  and  the  synchronized,  as  a  result  from  both  the  change  in  halfwidth  and  a 
shift  in  the  mean  value.  The  following  table  displays  the  decision  results  among  the 
MOEs  and  identifies  any  changes  in  the  decisions  between  the  base  and  synchronized 


results.  Each  of  the  CIs  developed  for  the  MOEs  below  are  calculated  at  the  a  =  0.05 
level  and  are  calculated  separately. 
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Table  4.26:  Control  Variate  and  Common  Random  Number  Configuration  Decision 


MOE 

BASE 

DECISION 

CV/CRN 

DECISION 

A-10LPS 

DIFFERENT 

DIFFERENT 

A- 6  LPS 

DIFFERENT 

DIFFERENT 

BLUE  FPS 

DIFFERENT 

DIFFERENT 

BLUE  LPS 

DIFFERENT 

DIFFERENT 

BLUE  SMG 

DIFFERENT 

DIFFERENT 

DHAHRAN  FPS 

DIFFERENT 

DIFFERENT 

F- 15  LPS 

NOT  DIFFERENT 

DIFFERENT 

FA- 18  FPS  total 

DIFFERENT 

DIFFERENT 

FA- 18  FPS  DCA 

NOT  DIFFERENT 

DIFFERENT 

MIG-23  LPS 

NOT  DIFFERENT 

NOT  DIFFERENT 

MIG-29  LPS 

DIFFERENT 

DIFFERENT 

MIRAGE  FPS  total 

DIFFERENT 

NOT  DIFFERENT 

MIRAGE  FPS  INT 

NOT  DIFFERENT 

NOT  DIFFERENT 

MIRAGE  LPS 

NOT  DIFFERENT 

NOT  DIFFERENT 

MUDAYSIS  FPS 

DIFFERENT 

DIFFERENT 

RED  FPS 

DIFFERENT 

DIFFERENT 

RED  LPS 

DIFFERENT 

DIFFERENT 

RIYADH  FPS 

DIFFERENT 

DIFFERENT 

SHAIBAH  FPS 

DIFFERENT 

DIFFERENT 

Looking  at  the  table  above,  we  can  see  that  the  decision  changes  three  times  and  they  are 


highlighted  in  gray. 


Since  the  change  is  not  consistently  positive  or  negative  and  the  MOEs  were 


selected  to  reflect  a  range  of  aggregation,  it  seems  logical  to  see  if  there  are  any 


commonalities  between  the  aggregation  levels  and  reduction.  When  sorted  into  the 


different  aggregation  levels  no  pattern  emerges.  The  only  item  of  possible  note  is  that  the 


data  collected  at  the  squadron  level  are  all  positive.  When  the  data  is  sorted  by  force, 
RED  and  BLUE,  there  is  also  no  notable  pattern. 


4.6.  Random  Number  Generator  Comparison 

To  perform  the  CRN  analysis  a  new  RNG  needed  to  be  implemented.  The 


research  in  generating  this  new  RNG  proclaims  that  it  has  better  statistical  properties  and 


its  speed  is  faster  (16).  For  the  application  in  THUNDER,  a  more  important  question  is 
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not  how  good  the  statistical  properties  are,  but  are  the  results  statistically  different  from 
the  original  RNG  and  do  replications  using  the  new  RNG  take  longer  to  run.  To  test 
these  questions,  three  different  comparisons  were  conducted.  To  perform  the 
comparisons,  paired-t  CIs  were  developed  for  all  MOEs  at  the  a  =  0.05  level.  Since  all 
MOEs  and  time  are  compared  simultaneously,  application  of  the  Bonferroni  inequality  is 


used  to  maintain  the  a  =  0.05  level  throughout  all  MOEs  and  time  comparisons. 


Table  4.27:  Random  Number  Experiment  Settings 


RNG 

SYNCHRONIZED 

ANTITHETIC 

COLLECT  C V 

CONFIGURATION 

SET 

(NEW/OLD) 

(YES/NO) 

(YES/NO) 

(YES/NO) 

(ORIGINAL/MODIFIED) 

B 

NEW 

NO 

NO 

NO 

ORIGINAL 

G 

OLD 

NO 

NO 

NO 

ORIGINAL 

H 

NEW 

YES 

NO 

NO 

ORIGINAL 

The  table  above  displays  the  experiment  settings  used  for  the  comparisons; 
remember  that  each  set  was  performed  with  300  replications.  The  first  subsection 
contains  a  comparison  between  the  original  RNG  and  the  new  RNG.  This  comparison 
uses  sets  B  and  G  from  the  table  above.  The  next  subsection  contains  a  comparison 
between  the  original  RNG  and  the  new  RNG  with  synchronized  random  inputs. 
Generating  a  unique  stream  for  each  random  input.  This  comparison  uses  sets  G  and  H 
from  the  above  table.  The  last  subsection  contains  a  comparison  between  the  new  RNG, 
not  synchronized,  and  the  new  RNG  with  synchronized  random  inputs.  This  comparison 
uses  sets  B  and  H  from  the  table  above. 

4.6.1.  Original  vs.  New  (not  synchronized)  Random  Number  Generator. 

Confidence  intervals  were  created  for  a  comparison  to  decide  if  the  new  RNG  is 
statistically  different  from  the  original  RNG.  The  following  table  and  graph  depicts  the 
confidence  intervals  and  determines  which,  if  any,  are  statistically  different. 
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Table  4.28:  Original  vs.  New  (not  synchronized)  Random  Number  Generator 


MOE 

LOWER 

BOUND 

UPPER 

BOUND 

CONTAINS 

ZERO 

A-10LPS 

-0.000673185 

0.000498496 

YES 

A- 6  LPS 

-0.002034542 

0.001989532 

YES 

BLUE  FPS 

-0.004493169 

0.00452297 

YES 

BLUE  LPS 

-0.000380992 

0.000266721 

YES 

BLUE  SMG 

-206.0933947 

143.946728 

YES 

DHAHRAN  FPS 

-0.013936772 

0.016032878 

YES 

F-111  LPS 

-0.000871293 

0.000863446 

YES 

F- 15  LPS 

-0.000541279 

0.00066419 

YES 

FA- 18  FPS  total 

-0.009697343 

0.006429048 

YES 

FA- 18  FPS  DCA 

-0.004787346 

0.000840694 

YES 

MIG-23  LPS 

-0.01238682 

0.013461924 

YES 

MIG-29  LPS 

-0.013436245 

0.018160475 

YES 

MIRAGE  FPS  total 

-0.011752971 

0.007611029 

YES 

MIRAGE  FPS  INT 

-0.009436666 

0.015837388 

YES 

MIRAGE  LPS 

-0.006410013 

0.011816126 

YES 

MUDAYSIS  FPS 

-0.012006375 

0.009409523 

YES 

RED  FPS 

-0.005130225 

0.002138004 

YES 

RED  LPS 

-0.002755065 

0.004516344 

YES 

RIYADH  FPS 

-0.002162778 

0.004127981 

YES 

SHAIBAH  FPS 

-0.016117409 

0.009644726 

YES 

TIME 

-0.184111452 

0.070778119 

YES 
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Figure  4.6:  Original  vs.  New  (not  synchronized)  Random  Number  Generator 

Confidence  Intervals 


Notice  how  all  of  the  confidence  intervals  contain  zero.  Statistically,  there  is  no 
difference  in  the  output  fonn  the  original  RNG  and  the  new  (not  synchronized)  RNG. 
4.6.2.  Original  vs.  New  (synchronized)  Random  Number  Generators. 
Confidence  intervals  were  created  for  a  comparison  to  decide  if  the  new  RNG, 
when  synchronized  for  CRN,  is  statistically  different  from  the  original  RNG.  The 
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following  table  and  graph  depicts  the  confidence  intervals  and  determines  which,  if  any, 
are  statistically  different. 


Table  4.29:  Original  vs.  New  (synchronized)  Random  Number  Generator 


MOE 

LOWER 

BOUND 

UPPER 

BOUND 

CONTAINS 

ZERO 

A- 10  LPS 

-0.000776407 

0.000343213 

YES 

A-6  LPS 

-0.002242076 

0.00172037 

YES 

BLUE  FPS 

-0.004762879 

0.003842456 

YES 

BLUE  LPS 

-0.000279397 

0.000398031 

YES 

BLUE  SMG 

-160.3646513 

186.0646513 

YES 

D HAH  RAN  FPS 

-0.012801006 

0.016895777 

YES 

F-111  LPS 

-0.000626852 

0.001139557 

YES 

F-15  LPS 

-0.000401774 

0.000842377 

YES 

FA-18  FPS  total 

-0.009003699 

0.006424452 

YES 

FA- 18  FPS  DCA 

-0.003217184 

0.002117636 

YES 

MIG-23  LPS 

-0.016939633 

0.008871754 

YES 

MIG-29  LPS 

-0.01958925 

0.009883308 

YES 

MIRAGE  FPS  total 

-0.011330577 

0.006628967 

YES 

MIRAGE  FPS  INT 

-0.008677052 

0.014232686 

YES 

MIRAGE  LPS 

-0.010454198 

0.00760329 

YES 

MUDAYSIS  FPS 

-0.007141542 

0.012470464 

YES 

RED  FPS 

-0.003830793 

0.003359536 

YES 

RED  LPS 

-0.003990189 

0.002962869 

YES 

RIYADH  FPS 

-0.002425742 

0.003669871 

YES 

SHAIBAH  FPS 

-0.017991997 

0.007191441 

YES 

TIME 

-0.219334662 

0.052667995 

YES 
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Figure  4.7:  Original  vs.  New  (synchronized)  Random  Number  Generator 

Confidence  Intervals 


Notice  how  all  of  the  confidence  intervals  contain  zero.  Statistically,  there  is  no 
difference  in  the  output  fonn  the  original  RNG  and  the  new  (synchronized)  RNG. 

4.6.3.  New  (not  synchronized)  vs.  New  (synchronized)  Random  Number 


Generators. 


Confidence  intervals  were  created  for  a  comparison  to  decide  if  the  new  RNG, 
when  synchronized  for  CRN,  is  statistically  different  from  the  new  RNG,  when  not 
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synchronized  for  CRN.  The  following  table  and  graph  depicts  the  confidence  intervals 
and  detennines  which,  if  any,  are  statistically  different. 


Table  4.30:  New  (not  synchronized)  vs.  New  (synchronized)  Random  Number 

Generator 


MOE 

LOWER 

BOUND 

UPPER 

BOUND 

CONTAINS 

ZERO 

A- 10  LPS 

-0.00074143 

0.000482925 

YES 

A-6  LPS 

-0.002085688 

0.001608992 

YES 

BLUE  FPS 

-0.004700458 

0.003750234 

YES 

BLUE  LPS 

-0.000235073 

0.000467977 

YES 

BLUE  SMG 

-122.4487384 

210.295405 

YES 

D HAH  RAN  FPS 

-0.01372776 

0.015726424 

YES 

F-111  LPS 

-0.000617935 

0.001138486 

YES 

F-15  LPS 

-0.000473583 

0.000791275 

YES 

FA-18  FPS  total 

-0.007783706 

0.008472755 

YES 

FA- 18  FPS  DCA 

-0.001374783 

0.004221887 

YES 

MIG-23  LPS 

-0.016885089 

0.007742106 

YES 

MIG-29  LPS 

-0.022430556 
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YES 
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YES 
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YES 
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YES 

RED  FPS 
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YES 
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YES 

RIYADH  FPS 

-0.003442907 
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YES 

SHAIBAH  FPS 
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YES 

TIME 

-0.146257618 
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YES 
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Figure  4.8  New  (not  synchronized)  vs.  New  (synchronized)  Random  Number 

Generator  Confidence  Intervals 


Notice  how  all  of  the  confidence  intervals  contain  zero.  Statistically,  there  is  no 
difference  in  the  output  form  the  new  RNG  (not  synchronized)  and  the  new 
(synchronized)  RNG. 

The  following  chapter  concludes  the  analysis  from  this  chapter.  It  develops 
usable  results  from  this  analysis  and  presents  future  work  in  these  areas. 
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5.  Conclusion 


This  section  contains  all  of  the  conclusions  and  recommendations  from  this 
research.  It  begins  with  conclusions  derived  from  the  analysis  of  chapter  4.  It  then 
covers  some  recommendations  for  further  work  or  study. 

5.1.  Conclusions 

5.1.1.  Control  Variates. 

Of  all  the  variance  reduction  techniques  applied  to  THUNDER,  control  variates 
have  performed  the  best.  It  consistently  realized  a  reduction  in  the  halfwidths  for  all 
MOEs  studied,  is  relatively  easy  to  implement,  can  be  used  to  reduce  the  required  number 
of  replications,  and  give  valuable  insight  into  the  model.  The  consistency  in  the 
halfwidth  reduction  allows  a  postprocessor  to  automatically  apply  control  variates  and 
give  a  reduced  halfwidth  without  worry  of  possibly  increasing  the  halfwidth.  The  ease  of 
implementation  also  allows  a  postprocessor  to  apply  control  variates  without  the  direct 
monitoring  of  the  analyst.  With  a  halfwidth  goal,  use  of  control  variates  will  reduce  the 
number  of  replication  needed  and  in  turn  reduce  the  time  required  to  analyze  the  model. 
Use  of  the  insight  given  by  the  weight  and  sign  of  the  controls  can  help  identify 
inconsistencies  of  the  code  or  methodology.  It  can  also  aid  in  understanding  how 
elements  of  the  model  affect  desired  MOEs. 

The  analysis  in  chapter  4  identified  a  statistically  significant  increase  in  the 
replication  generation  time  for  control  variates.  This  difference  in  time  ranges  between 
0.01  and  0. 18  second.  The  size  of  the  difference  is  a  fraction  of  a  second  and  is  not 
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2 

practically  significant.  Also,  the  model  R"s  are  small  and  the  reduction  in  the  halfwidths 
are  small,  but  the  effects  of  the  reduction  may  be  significant  to  the  analysis.  Therefore, 
consistent  use  of  control  variates  is  practical  and  effective  in  light  of  the  drawbacks. 

5.1.2.  Common  Random  Numbers. 

When  comparing  different  configurations  of  a  system,  a  form  of  common  random 
numbers  is  generally  used,  either  voluntarily  or  nonvoluntarily.  When  two  or  more 
configurations  start  with  the  same  seeds  and  only  one  stream  is  accessed,  common 
random  numbers  are  used.  This  could  potentially  cause  unintentional  problems.  Since 
common  random  numbers  are  most  likely  being  used,  they  should  be  used  properly  where 
all  of  the  random  inputs  are  synchronized. 

In  this  research,  common  random  numbers  did  not  necessarily  reduce  all  of  the 
halfwidths.  In  the  19  MOEs  analyzed,  there  were  14  halfwidth  reductions  and  5 
halfwidth  increases.  When  looking  through  the  changes  in  the  halfwidths,  the  largest 
reduction  is  16.25%  with  a  total  of  7  MOEs  with  reductions  greater  than  7%,  while  the 
largest  increase  in  the  halfwidths  was  6.03%.  Therefore,  it  is  more  likely  to  get  a 
reduction  in  a  halfwidth  than  getting  an  increase  in  a  halfwidth.  Also,  the  size  of  a 
reduction  in  a  halfwidth  is  generally  larger  than  an  increase  in  a  halfwidth. 

5.1.3.  Antithetic  Variates. 

When  looking  at  the  results  from  antithetic  variates,  the  implementation  does  not 
seem  very  promising.  The  amount  of  extra  effort  in  generating  the  antithetic  pair  runs 
results  in  only  1 1  of  the  19  MOEs  analyzed,  57.89%,  receiving  a  reduction  in  the 
halfwidth. 
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5.1.4.  Control  Variates  and  Antithetic  Variates. 


This  method  is  a  little  more  promising  than  the  standard  antithetic  variates.  It  still 
requires  the  same  amount  of  effort  in  generating  the  antithetic  pair  runs  and  only  57.89% 
of  the  MOEs  analyzed  received  a  reduction  in  the  halfwidths.  The  major  difference  is 
that  the  1 1  reductions  in  halfwidths  are  larger  than  either  antithetic  variates  or  control 
variates  independently.  Also  the  8  increases  in  halfwidths  are  smaller  than  the  regular 
antithetic  variates.  This  implies  that  there  still  may  be  some  benefit  in  using  this  method 
if  the  MOE  is  determined  to  be  reducible  by  antithetic  variates.  Also,  since  control 
variates  are  applied  in  this  method,  insight  gained  from  analyzing  the  weight  and 
direction  of  the  controls  can  be  beneficial. 

5.1.5.  Control  Variates  and  Common  Random  Numbers. 

This  method  did  not  prove  very  successful.  Thirteen  of  the  19  or  68%  of  the 
MOEs  received  a  reduction  in  variance,  and  the  decision  changed  3  times.  The  CRN 
method  has  produced  mixed  results  in  the  past,  so  an  application  of  just  the  CVs  was 
examined.  The  analysis  of  the  CV  by  itself  did  prove  successful,  18  of  the  19  or  95%  of 
MOEs  received  a  reduction  in  the  halfwidth.  Therefore,  some  sort  of  application  of  only 
the  CV  method  may  be  beneficial  to  implement  in  two  system  comparisons. 

5.1.6.  Random  Number  Generator. 

To  properly  implement  common  random  numbers,  a  new  random  number 
generator  needed  to  be  implemented.  With  the  implementation  of  a  new  random  number 
generator,  we  needed  to  make  sure  the  new  random  number  generator  would  not 
significantly  alter  the  THUNDER  results.  Three  different  comparisons  were  performed. 
They  were  the  new  random  number  generator  without  synchronizing  all  random  inputs 
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against  the  original  random  number  generator,  the  new  random  number  generator 
synchronizing  all  random  inputs  against  the  original  random  number  generator,  and  the 
new  random  number  generator  without  synchronizing  all  random  inputs  against  the  new 
random  number  generator  synchronizing  all  random  inputs.  The  results  for  all  three 
analyses  were  the  same.  At  an  a  =  0.05  level,  all  MOEs  were  statistically  equal.  An 
important  part  of  the  test  is  that  the  replication  times  were  also  statistically  equal. 

5.2.  Recommendations  for  Further  Work  or  Study 

While  working  on  this  research  and  examining  the  THUNDER  source  code  some 
topics  for  further  work  or  study  were  discovered.  Some  of  them  are  small  fixes  and  some 
are  areas  where  an  analysis  is  needed. 

5.2.1.  Further  THUNDER  Work. 

First,  examination  of  the  random  variate  modules  of  THUNDER,  it  was  realized 
that  THUNDER  does  not  allow  the  analyst  to  assign  all  three  parameters  to  the  triangular 
distribution.  Fixing  this  problem  would  require  a  through  search  through  the  code  to 
identify  all  sections  of  code  that  are  associated  with  the  triangular  distribution. 

Second,  more  random  variate  distributions  should  be  added  to  THUNDER. 
Currently  THUNDER  only  allows  the  13  distributions  in  Table  3.2.  More  distributions 
would  allow  more  flexibility  for  the  analyst. 

Third,  incorporate  control  variates  and  antithetic  variates  into  the  post  processors. 
Application  should  be  simple,  by  following  the  methodology  used  in  this  research.  An 
output  of  control  variate  weights  should  also  be  incorporated  into  the  post  processors. 
Common  random  numbers  does  not  need  to  be  added  to  a  postprocessor  since  it  is  a 
passive  variance  reduction  technique. 
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Fourth,  a  table  identifying  the  random  inputs  and  what  they  are  used  for  should  be 
made  for  THUNDER  users.  This  could  be  added  as  an  appendix  to  the  THUNDER 
documentation  and  used  in  conjunction  to  control  variate  weights  after  it  is  incorporated 
in  a  post  processor. 

Fifth,  some  of  the  random  inputs  in  THUNDER  are  mislabeled.  While  using  the 
name  structucre  to  locate  the  random  inputs,  the  are  ..BSE030.1,  ..BSE001.1, 

..BSE001.2,  and  ..AIR007.1  were  not  in  the  proper  source  files.  After  a  thurogh  search 
through  every  source  files,  the  correct  source  files  were  identified.  Therefore,  they 
should  be  changed  to  ..UTL260.1,  ..BSE225.1,  ..BSE225.2,  and  ..AIR014.1  respectively. 

Sixth,  pennanently  implement  a  new  random  number  generator  in  THUNDER. 
The  random  number  generator  used  for  this  research  is  authorized  for  student  use  and 
authorization  from  Dr.  L’Ecuyer  is  needed  to  implement  MRG32k3a  for  commercial 
purposes.  It  is  also  possible,  with  a  little  research,  that  any  of  the  other  parameters  from 
Dr.  L’Ecuyers  article  (16)  could  be  used  to  build  a  combined  multiple  recursive  random 
number  generator  for  the  Air  Force. 

5.2.2.  Further  THUNDER  Study. 

First,  find  a  way  to  utilize  random  inputs  that  use  more  than  one  distribution. 
These  variates  are  not  used  in  the  current  analysis.  A  possible  direction  to  go  from  here 
is  to  apply  control  variates  at  the  central  random  number  draw  and  not  on  the  random 
inputs. 

Second,  test  control  variates,  common  random  numbers,  and  antithetic  variates 
with  larger  THUNDER  databases.  The  ME  database  is  relatively  small  and  this  may 
affect  the  response  of  the  variance  reduction  technique.  Also,  a  test  to  see  if  test  pilot 
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runs  of  a  small  amount  of  days  or  only  a  couple  of  replications  has  the  same  affect  as  a 
full  test  with  a  large  amount  of  days  or  replications. 

Third,  try  applying  antithetic  variates  only  to  random  inputs  that  are  not  branches 
and  common  random  numbers  to  branches.  This  method  may  make  antithetic  variates 
more  effective.  Then,  combined  with  control  variates,  this  method  may  have  a  lot  of 
variance  reduction  potential. 
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Appendix  A:  THUNDER  Report  Example  (1  Full  Day) 
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Appendix  B:  Table  of  Distribution  Expected  Values 


Distribution 

Probability  Function 

Expected  Value 

Beta 

f(y)=  \T(<*  +  P) 

0<y  < 

v'n-vr 

l 

a 

a  +  /3 

Binomial 

p{y)= 

y 

fn\  , 

py(l-p)"y 

\y) 

=  0, 

np 

Erlang 

f(y)=  Ae  A 
o  <y 

A  V-1 

UJ 

(«-!)! 

k/3 

Exponential 

0  <  /?;  0  <y<co 

P 

Gamma 

f(y)= 

1  1  — 
r(«)/r_T  e 

0  <  V  <  oo 

af5 

Logorithmic  Normal 

f{y)=  /r-J 

ycryiTt 

o  <y 

H+cr 2 

e  2 

Normal 

Ay)-  pz~^ 

crpln 
-  co  <  y  < 

+oo 

P 

Poisson 

1  y  —A 

pb’)=A 

y- 

y  =  0,1,2,... 

1 

Randi 

Discrete  Uniform  (a,  b) 

M=(b-a  +  i) 
y  =  a,  a  +  \,...,b-l,b 

b 

I' 

i=a 

(b-a  +  i) 
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Distribution 

Probability  Function 

Expected  Value 

Random 

Uniform  (0, 1) 

f(y)=\ 

0<y<l 

0.5 

Triangle 

f(y)= 

2(y~a)  ^  ^ 

(b-alc-a)  a~y~C 

2  (b-y) 

7 — V  w  7  s.  c<v<b 
{b  -  a  )(b  -c  ) 

a  <  y  <b 

a  +  b  +  c 

3 

Uniform 

e,<y<e2 

6\  +d2 

2 

Weibull 

f(y)  =  ap-‘y‘-'e ^ 

0<y 

(  n 

/T  1  +  - 

V  a) 
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Appendix  C:  The  Modified  and  Added  Files 


The  modified  SIMSCRIPT  II. 5  files  located  in  the  Source  directory: 


ADF040.sim 

ADF041  .sim 

ADF101  .sim 

ADF105.sim 

ADF121  .sim 

ADF140.sim 

ADF150.sim 

ADF160.sim 

ADF161  .sim 

ADF200.sim 

ADF420.sim 

ADF430.sim 

ADF440.sim 

ADF452.sim 

ADF460.sim 

ADF500.sim 

AIR014.sim 

AIR017.sim 

AIR021  .sim 

AIR041.sim 


AIR042.sim 
AIR050.sim 
AIR051  .sim 
AIR060.sim 
AIR070.sim 
AIR101  .sim 
AIR1 10. sim 
AIR120.sim 
AIR150.sim 
AIR241  .sim 
AIR526.sim 
AIR528.sim 
AIR540.sim 
AIR550.sim 
AIR561  .sim 
AIR564.sim 
AIR567.sim 
AIR602.sim 
AIR800.sim 
AIR810.sim 


AIR830.sim 

AIR840.sim 

AIR91 1  .sim 

AIR920.sim 

ANT21 1  .sim 

BSE001  .sim 

BSE002.sim 

BSE003.sim 

BSE020.sim 

BSE050.sim 

BSE200.sim 

BSE201  .sim 

BSE225.sim 

BSE251  .sim 

BSE260.sim 

BSE270.sim 

BSE400.sim 

BSE800.sim 

GRD095.sim 

GRDIOO.sim 


GRD300.sim 

GRD302.sim 

GRD309.sim 

GRD871  .sim 

ISROOO.sim 

ISR001  .sim 

ISR004.sim 

ISR005.sim 

ISR1 15. sim 

ISR250.sim 

ISR251  .sim 

ISR360.sim 

ISR365.sim 

ISR400.sim 

ISR500.sim 

LOG700.sim 

PLAOIO.sim 

PLA443.sim 

PLA557.sim 

REC01 1  .sim 


The  added  SIMSCRIPT  II. 5  files  located  in  the  Source  directory: 

ZEARLI.sim 

ZEARL2.sim 


The  modified  THUNDER  scripts  located  in  the  bin  directory: 

ttrun 

ttrep 

ttcompile 

ttuser.each 


The  modified  ME  data  files  located  in  the  Data/ME  directory: 
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REC012.sim 
REC013.sim 
REC014.sim 
REC015.sim 
REC016.sim 
REC017.sim 
REC018.sim 
REC310.sim 
REC320.sim 
REC330.sim 
REC341  .sim 
REC900.sim 
UTL104.sim 
UTL260.sim 
UTL305.sim 
UTL520.sim 
UTL550.sim 
UTL750.sim 
UTL920.sim 
STA900.sim 


control.dat 


The  added  ME  data  files  located  in  the  Data/ME  directory: 

newseed.val 

The  added  C  files  located  in  the  Source  directory: 

RngStream.c 

RngStream.h 
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Appendix  D:  CONTROL.DAT  File 


CONTROLS . 020 

REPORT. TITLE  "ME" 


BASIC. SIMULATION. CONTROLS 

NUMBER . OF . DAYS . OF . WAR ( INT . DAYS )  5 

BEGIN. DAYLIGHT (DEC. HOURS)  6.00 

END. DAYLIGHT (DEC. HOURS)  18.00 

ADX . PROVIDE . TERM. AD . FOR. FIXED . TGTS  YES 

IGNORE. RANGE. CONSTRAINTS  NO 

COUNT. ALERT. MSN. AS. COMPLETE  NO 

END. BASIC. SIMULATION. CONTROLS 

GAME. CONTROLS 

GAME.  FILE.  MOD.  FLAG  YES 

GAME . ENABLE  NO 

GAME. CYCLE  24 

FIRST. GAME. STOP (DAYS)  2.0 

END . GAME . CONTROLS 

ADF . CALIBRATION . CONTROLS 

ADF. CAL I BRAT I ON. MODE. ENABLE  NO 


@  if  enabled 

@  only  preplanned  ATOs  fly 

@  all  ADvsAC  Pks  =  0.0 

@  unlimited  AD  ammo  reloads 

@  lethal  SEAD  disabled 

@  air-to-ground  disabled 

@  special  ADF  calibration  transactions  enabled 

@  ADF  results  averaged  over  multiple  reps: 


REPLICATIONS . PER. ADF . CALCULATION  500 

USE. MANUAL. IADS. VALUES (YES, NO)  NO 

MANUAL . IADS . INTEGRATION . LEVEL (0. 0-1.0)  0.7 

MANUAL . IADS . SECONDS . DELAY  15.0 

MANUAL. IADS. INTIMIDATE  0.0 

END .ADF . CALIBRATION . CONTROLS 

COMPUTATIONAL . RESOLUTION . LEVELS (LOW, HIGH, VERY . HIGH) 
BLUE. AIR. DEFENSE  HIGH 

RED. AIR. DEFENSE  HIGH 

AIR. WAR  HIGH 

BLUE .RECONNAISSANCE  HIGH 

RED. RECONNAISSANCE  HIGH 

WEATHER  HIGH 

GROUND . MOVEMENT  HIGH 

CRITICAL. RESOURCES  HIGH 

END . COMPUTATIONAL . RESOLUTION . LEVELS 

GRAPHIC . OUTPUT . CONTROLS 

WRITE. GRAPHIC. TO. TRANS  NO 
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POST. PROCESS. GRAPHICS (BOTH, CHART, MAP, NONE)  BOTH 
AD. COMPLEX. STATUS. UPDATE. CYCLE (HOURS)  6 

AIRBASE . STATUS . UPDATE . CYCLE (HOURS )  3 

CHOKE. POINT. STATUS. UPDATE. CYCLE (HOURS)  6 

COMMAND. STATUS. UPDATE. CYCLE (HOURS)  6 

LOG. FAC. STATUS. UPDATE. CYCLE (HOURS)  6 

STRATEGIC. TARGET. STATUS. UPDATE. CYCLE (HOURS)  6 
SUPPLY. TRAIN. STATUS. UPDATE. CYCLE (HOURS)  6 

TBM. UNIT. STATUS. UPDATE. CYCLE (HOURS)  6 

UNIT. STATUS. UPDATE. CYCLE (HOURS)  6 

PERCEPT. STATUS. UPDATE. CYCLE (HOURS)  6 

AIR. REPORT. CYCLE (DAY, GAME, BLUE, RED)  DAY 

END . OUTPUT . CONTROLS 

OVERALL . REPORT . CONTROLS 

REPORT. MODE ( STANDARD, METRIC, BOTH)  STANDARD 

END . OVERALL . REPORT . CONTROLS 


DATA . REPORT . CONTROLS 

NUMBER . OF . DATA . RE  PORT  S 

CONTROL. FILE 

RANDOM. SEEDS 

STANDARD . TARGETS 

STANDARD . TARGET . REPAIR. PROBS 

TYPE. RADAR 

TYPE . JAMMER 

AIR. DEFENSE . DATA. REPORTS 
TYPE. AIR. DEFENSE 
POSTURES . & . CLASSES 
THREAT. CLASSES 
INTEGRATED. AD. SYSTEM 
ADA. ENGAGEMENT . PROBS 
AD. VS .AIRCRAFT . PKS 
AD. COMPLEXES 
PLANNING . DATA . REPORTS 
AIR. RULES 

AIR. PLANNING. RULES 
AIR. ALLOCATION 
AIR. PLANNING. FACTORS 
OCA. TARGETS 
PREPLANNED. ATOS 
PREPLANNED . LIFT .ATOS 
ZONE . SECTOR. TGT . PRIORITY 
INT . DEPTH . FACTOR. CURVES 
STI .TARGETS 

AIR. TO. AIR. DATA. REPORTS 
AIR. TO. AIR. PKS 
RELEASE . RANGE . ADVANTAGE 
CONFIG . DETECT . PROBS 
AIR. TO. GROUND. PKS 
HARM. PKS 
MINE. PKS 

TAKEOFF . LANDING . PKS 

WMD. EFFECTS 

TYPE. AIR. MUNITIONS 


D.2 


91 

DR-2 

NO 

DR- 3 

NO 

DR- 5 

NO 

DR- 6 

NO 

DR- 10 

NO 

DR- 11 

NO 

k  k  k  k 

DR- 12 

NO 

DR- 13 

NO 

DR- 16 

NO 

DR- 14 

NO 

DR- 15 

NO 

DR-27 

NO 

DR- 6  6 

NO 

k  k  k  k 

DR-20 

NO 

DR- 7  9 

NO 

DR-21 

NO 

DR-67 

NO 

DR-23 

NO 

DR-2  9 

NO 

DR-22 

NO 

DR-65 

NO 

DR- 6  8 

NO 

DR- 7  0 

NO 

k  k  k  k 

DR-24 

NO 

DR-25 

NO 

DR-2  6 

NO 

DR-2  8 

NO 

DR- 4  0 

NO 

DR- 41 

NO 

DR- 4  2 

NO 

DR- 4  3 

NO 

DR- 31 

NO 

MUNITION. SURVIVABILITY. CURVES 

DR- 

44 

NO 

TYPE .AIRCRAFT 

DR- 

32 

NO 

AIRCRAFT . MAINTENANCE 

DR- 

39 

NO 

TBM. UNITS 

DR- 

75 

NO 

TBM. DETECTION 

DR- 

76 

NO 

AIRBASE 

DR- 

33 

NO 

CARRIER . BATTLE . GROUP 

DR- 

30 

NO 

SUPPLY. KIT 

DR- 

34 

NO 

SQUADRON 

DR- 

35 

NO 

AIR. TO. GRD. MIN. CEIL. VIS 

DR- 

36 

NO 

SUPPLY . URGENCY . CURVES 

DR- 

37 

NO 

CRITICAL . RESOURCES 

DR- 

38 

NO 

NODES 

DR- 

45 

NO 

TRANS . SHIPMENT . POINTS 

DR- 

46 

NO 

ARCS 

DR- 

47 

NO 

CHOKE . POINTS 

DR- 

48 

NO 

MINE . DELAY . CURVES 

DR- 

49 

NO 

BATTLEFIELD 

DR- 

50 

NO 

AIR. NETWORK 

DR- 

73 

NO 

GROUND . TARGE  T. ACQUISITION 

DR- 

51 

NO 

TARGE  T. ACQUISITION. CURVE  S 

DR- 

17 

NO 

TERRAIN 

DR- 

52 

NO 

INTERVISIBILITY. CURVES 

DR- 

53 

NO 

TYPE. C3. FACILITY 

DR- 

55 

NO 

COMMAND 

DR- 

56 

NO 

GROUND. RULES 

DR- 

57 

NO 

SUPPLY . TRAIN . TARGETS 

DR- 

54 

NO 

LOGISTIC. FACILITY 

DR- 

58 

NO 

TYPE. UNIT 

DR- 

59 

NO 

TYPE. EQUIPMENT 

DR- 

60 

NO 

EQUIPMENT .SIZE. CATEGORY 

DR- 

61 

NO 

WEAPON. VS .EQUIPMENT .EFFECTS 

DR- 

62 

NO 

UNIT 

DR- 

63 

NO 

LIFT. EVENTS 

DR- 

82 

NO 

LOG. RULES 

DR- 

81 

NO 

SPACE. LOGISTICS 

DR- 

78 

NO 

PERCEPTIONS 

DR- 

64 

NO 

STRATEGIC . TARGETS 

DR- 

69 

NO 

AIRCRAFT . FACTORS 

DR- 

71 

NO 

GROUND . WAR . SUPPORT . EFFECTS 

DR- 

72 

NO 

BOOST . PHASE . INTERCEPT 

DR- 

77 

NO 

TARGET. AVAILABILITY 

DR- 

74 

NO 

WEATHER. STATIONS 

DR- 

80 

NO 

SATELLITE .NAV. JAM. STATES 

DR- 

92 

NO 

SATELLITE .NAV.MUNT. EFFECTS 

DR- 

93 

NO 

SATELLITE . PASSES 

DR- 

94 

NO 

I SR. DATA. REPORTS 

•k  -k  -k 

* 

ISR. SATELLITES 

DR- 

83 

NO 

I SR. PLANNING 

DR- 

84 

NO 

ISR. TARGETS 

DR- 

85 

NO 

ISR. SENSORS .AND. EMITTERS 

DR- 

86 

NO 

ISR. EFFECTS 

DR- 

87 

NO 

ISR. INITIALIZATION 

DR- 

CO 

CO 

NO 

ISR. EVENTS 

DR- 

89 

NO 

ISR. NODES 

DR- 

90 

NO 
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ISR. SENSOR. VS .EMITTER. PROBS 

DR- 91 

NO 

MUNITION .MANAGEMENT . FACTORS 

DR-95 

NO 

CHEMICAL . WEAPON . EFFECTS 

DR- 9  6 

NO 

CHEMICAL .AGENT . DELIVERY 

DR-97 

NO 

CAMPAIGN . COST . DATA 

DR- 9  9 

NO 

END . DATA . REPORT . CONTROLS 

OUTPUT . REPORT . CONTROLS 

COMPLETED. MISSION. INFO 

AW- 1 

NO 

BOTH 

0.00 

300.00 

DAILY. AIR. WAR. RESULTS . SUMMARIES 

k  k  k  k 

MISSION. & . SORTIE . SUMMARY 

AW- 2 

NO 

BOTH 

0.00 

300.00 

AIRCRAFT . LOSS . SUMMARY 

AW- 3 

YES 

BOTH 

0.00 

300.00 

AIRBASE .ACTIVITY 

AW- 4 

NO 

BOTH 

0.00 

300.00 

SORTIE . GENERATION 

AW- 5 

YES 

BOTH 

0.00 

300.00 

CANCELLED. MISSION. INFO 

AW- 6 

NO 

BOTH 

0.00 

300.00 

INDIVIDUAL. MISS ION. KILL. RESULTS 

k  k  k  k 

DCA. MISSION. KILLS 

AW- 7 

NO 

BOTH 

0.00 

300.00 

ODCA. MISS ION. KILLS 

AW- 8 

NO 

BOTH 

0.00 

300.00 

HVAA. MISS ION. KILLS 

AW- 9 

NO 

BOTH 

0.00 

300.00 

BARCAP. MISS ION. KILLS 

AW- 10 

NO 

BOTH 

0.00 

300.00 

FSWP. MISS ION. KILLS 

AW- 11 

NO 

BOTH 

0.00 

300.00 

DTBM. MISS ION. KILLS 

AW- 12 

NO 

BOTH 

0.00 

300.00 

STI .MISSION. KILLS 

AW- 13 

NO 

BOTH 

0.00 

300.00 

CAS .MISSION. KILLS 

AW- 14 

NO 

BOTH 

0.00 

300.00 

BAI .MISSION. KILLS 

AW- 15 

NO 

BOTH 

0.00 

300.00 

INT. MISSION. KILLS 

AW- 16 

NO 

BOTH 

0.00 

300.00 

OCA. MISSION. KILLS 

AW- 17 

NO 

BOTH 

0.00 

300.00 

DSEAD. MISS ION. KILLS 

AW- 18 

NO 

BOTH 

0.00 

300.00 

SUPPRESS ION. MISS ION. KILLS 

AW- 19 

NO 

BOTH 

0.00 

300.00 

OTBM. MISS ION. KILLS 

AW- 2  0 

NO 

BOTH 

0.00 

300.00 

AIR. ESCORT. MISS ION. KILLS 

AW- 21 

NO 

BOTH 

0.00 

300.00 

HIGH .VALUE .ASSET .ACTIVITY 

AW- 2  2 

NO 

BOTH 

0.00 

300.00 

DEFENDING. AIRCRAFT. KILLS 

AW- 2  3 

NO 

BOTH 

0.00 

300.00 

LIFT. MISSION 

AW- 2  4 

NO 

BOTH 

0.00 

300.00 

AIRCRAFT . MAINTENANCE 

k  k  k  k 

TYPE. AIRCRAFT. MX. ACTIVITY 

MX- 1 

NO 

BOTH 

0.00 

300.00 

CONSUM. RESOURCE. ACTIVITY 

MX-2 

NO 

BOTH 

0.00 

300.00 

NON. CONSUM. RES .ACTIVITY 

MX- 3 

NO 

BOTH 

0.00 

300.00 

GROUND . COMBAT . CYCLE . REPORTS 

k  k  k  k 

UNIT. STATUS 

CC-1 

NO 

BOTH 

0.00 

300.00 

C3 . FACILITY . STATUS 

CC-2 

NO 

BOTH 

0.00 

300.00 

CHOKE . POINT . STATUS 

CC-3 

NO 

BOTH 

0.00 

300.00 

FLOT. MOVEMENT 

CC-4 

NO 

BOTH 

0.00 

300.00 

AIR. GRID. AD. STATUS 

CC-5 

NO 

BOTH 

0.00 

300.00 

TYPE .AD. SITE . STATUS 

CC-6 

NO 

BOTH 

0.00 

300.00 

STRATEGIC . TARGET . STATUS 

CC-1 

NO 

BOTH 

0.00 

300.00 

IADS . OBJECT . STATUS 

CC-8 

NO 

BOTH 

0.00 

300.00 

AIR. MISSION. PLANNING. REPORTS 

k  k  k  k 

APPORTIONMENT .ALLOCATION 

MP-1 

NO 

BOTH 

0.00 

300.00 

AIR . TASKING . ORDERS 

MP-2 

NO 

BOTH 

0.00 

300.00 

SORTIES . SCHEDULED . SUMMARY 

MP-3 

NO 

BOTH 

0.00 

300.00 

TARGET . PRIORITIES 

MP-4 

NO 

BOTH 

0.00 

300.00 

AIR. REFUELING. ALLOCATION 

MP-5 

NO 

BOTH 

0.00 

300.00 

AIR. PLANNING. RULE . STATUS 

MP-6 

NO 

BOTH 

0.00 

300.00 

GROUND . SUPPLY . CYCLE . REPORTS 

k  k  k  k 
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LOGISTIC . FACILITY . STATUS 

SC-1 

NO 

BOTH 

0.00 

300.00 

SUPPLY . TRAIN . STATUS 

SC-2 

NO 

BOTH 

0.00 

300.00 

CRITICAL . RESOURCE . STATUS 

SC-3 

NO 

BOTH 

0.00 

300.00 

SPACE. LOGISTICS .REPORT 

SL-1 

NO 

BOTH 

0.00 

300.00 

END . OUTPUT . REPORT . CONTROLS 
GAME . REPORT .SELECTION 

GC-1  NO  BOTH  "AIR  COMMAND  MISSION  ALLOCATION  REPORT" 

GC-2  NO  BOTH  "AIR  COMMAND  RESOURCE  ALLOCATION  REPORT" 

GC-3  NO  BOTH  "TARGET  PRIORITY  REPORT" 

GC-4  NO  BOTH  "GROUND  COMMAND  STRUCTURE  WITH  SUPPORTING  AIR 
COMMANDS" 

GC-5  NO  BOTH  "AIR  COMMAND  STRUCTURE  WITH  SUPPORTED  GROUND  COMMANDS" 
GC-6  NO  BOTH  "AIR  BASE  ASSETS  REPORT" 

GC-7  NO  BOTH  "LOGISTICS  FACILITY  ASSETS  REPORT" 

GC-8  NO  BOTH  "AIRCRAFT  CHARACTERISTICS  REPORT" 

GC-9  NO  BOTH  "ZONE  SECTOR  PERCEPTION  HISTORY" 

GC-10  NO  BOTH  "FLOT  MOVEMENT  REPORT" 

GC-11  NO  BOTH  "MISSION  ALLOCATION  PROJECTION" 

GC-12  NO  BOTH  "PROJECTED  AIR  TASKING  ORDERS" 

GC-1 3  NO  BOTH  "PROJECTED  SORTIES  SCHEDULED" 

GC-1 4  NO  BOTH  "AIRCRAFT  PLANNING  FACTORS" 

GC-1 5  NO  BOTH  "PREPLANNED  ATO  REPORT" 

GC-1 6  NO  BOTH  "AIR  GRID  AIR  DEFENSE  SITE  STATUS" 

GC-1 7  NO  BOTH  "AIR  COMBAT  RULES" 

GC-1 8  NO  BOTH  "AIR  PLANNING  RULES" 

GC-1 9  NO  BOTH  "PREPLANNED  LIFT  ATO  REPORT" 

GC-20  NO  BOTH  "MUNITION  MANAGEMENT  FACTORS" 

END . GAME . REPORT . SELECTION 


TRANSACTION . CONTROLS 
0DEBUG. TRANSACTIONS 


ADFL 

0 

0.000 

300.00 

ADFH 

0 

0.000 

300.00 

AIRA 

0 

0.000 

300.00 

AIRF 

0 

0.000 

300.00 

AIRG 

0 

0.000 

300.00 

AIRP 

0 

0.000 

300.00 

ANTC 

0 

0.000 

300.00 

ANTI 

0 

0.000 

300.00 

ANTP 

0 

0.000 

300.00 

BSEL 

0 

0.000 

300.00 

BSEM 

0 

0.000 

300.00 

BSEO 

0 

0.000 

300.00 

CONA 

0 

0.000 

300.00 

GRDC 

0 

0.000 

300.00 

GRDM 

0 

0.000 

300.00 

GRDO 

0 

0.000 

300.00 

I SRE 

0 

0.000 

300.00 

ISRP 

0 

0.000 

300.00 

I SRR 

0 

0.000 

300.00 

ISRS 

0 

0.000 

300.00 

I SRT 

0 

0.000 

300.00 

LOGS 

0 

0.000 

300.00 

LOGE 

0 

0.000 

300.00 
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NETM 

0 

0.000 

300.00 

NETP 

0 

0.000 

300.00 

PLAA 

0 

0.000 

300.00 

PLAT 

0 

0.000 

300.00 

RECT 

0 

0.000 

300.00 

RULP 

0 

0.000 

300.00 

SATL 

0 

0.000 

300.00 

SATO 

0 

0.000 

300.00 

STAA 

0 

0.000 

300.00 

UTLA 

0 

0.000 

300.00 

UTLG 

0 

0.000 

300.00 

UTLF 

0 

0.000 

300.00 

WTHH 

0 

0.000 

300.00 

0GRAPHIC. TRANSACTIONS 


GRAA  3 

0. 

,000  300. 

.00 

GRAG  4 

0. 

,000  300. 

.  00 

GRAI  2 

0. 

,000  300. 

.00 

@ ANALYSIS . 

TRANSACTIONS 

EQ. KILLS 

0 

0.000 

300.000 

AA. KILLS 

0 

0.000 

300.000 

SA. KILLS 

0 

0.000 

300.000 

MUNT.EXP 

0 

0.000 

300.000 

STR.TGT 

0 

0.000 

300.000 

AA.  ENC 

0 

0.000 

300.000 

@ DATABASE . 

TRANSACTIONS 

DB . PLAN 

0 

0.000 

300.000 

DB.EXEC 

0 

0.000 

300.000 

DB. STATE 

0 

0.000 

300.000 

@RMS . TRANSACTIONS 

RMSPP 

0 

0.000 

300.000 

@MP. TRANSACTIONS 

MPPP 

0 

0.000 

300.000 

@AIR. LIFT . 

TRANSACTIONS 

ALFT 

0 

0.000 

300.000 

@ I SR . TRANSACTIONS 

BPIPP 

0 

0.000 

300.000 

ISRPP 

0 

0.000 

300.000 

ISRPP2 

0 

0.000 

300.000 

@ SURVIVE . TRANSACTIONS 

SURVIVE 

0 

0.000 

300.000 

0WARGAME . TRANSACTIONS 

WARGAME 

0 

0.000 

300.000 

0GRAPHER. 2000. TRANSACTIONS 
GRAPHER2K  0  0.000  300.000 


END . TRANSACTION . CONTROLS 


VARIANCE . REDUCTION . CONTROLS 

USE .NEW. RANDOM. NUMBER. GENERATOR  YES 

USE. COMMON. RANDOM. NUMBERS  YES 

USE. ANTITHETIC. VARIABLES  NO 

USE. CONTROL. VARIATES  YES 

RANDOM . NUMBER . STREAM . DATA 

READ. RANDOM. SEEDS  YES 
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WRITE . RANDOM. SEEDS 
SINGLE . SEED . ALL . NUMBERS 
SINGLE. SEED. IS (INT) 

RN. STREAMS 
ADF040.1  1 

ADF041.1  1 

ADF041.2  1 

ADF041.3  1 

ADF101.1  1 

ADF101.2  1 

ADF105.1  1 

ADF121.1  1 

ADF121.2  1 

ADF121.3  1 

ADF140.1  1 

ADF140.2  1 

ADF140.3  1 

ADF150.1  1 

ADF150.2  1 

ADF150.3  1 

ADF150.4  1 

ADF150.5  1 

ADF150.6  1 

ADF150.7  1 

ADF160.1  1 

ADF161.1  1 

ADF161.2  1 

ADF161.3  1 

ADF161.4  1 

ADF161.5  1 

ADF200.1  1 

ADF420.1  1 

ADF420.2  1 

ADF420.3  1 

ADF430.1  1 

ADF430.2  1 

ADF430.3  1 

ADF440.1  1 

ADF440.2  1 

ADF440.3  1 

ADF452.1  1 

ADF452.2  1 

ADF452.3  1 

ADF452.4  1 

ADF452.5  1 

ADF452.6  1 

ADF452.7  1 

ADF460.1  1 

ADF460.2  1 

ADF500.1  1 

AIR007.1  1 

AIR017.1  1 

AIR041.1  1 

AIR041.2  1 


YES 

NO 

1 
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AIR041.3  1 
AIR042.1  1 
AIR042.2  1 
AIR050.1  1 
AIR050.2  1 
AIR051.1  1 
AIR060.1  1 
AIR060.2  1 
AIR060.3  1 
AIR070.1  1 
AIR070.2  1 
AIR021.1  1 
AIR101.1  1 
AIR101.2  1 
AIR110.1  1 
AIR120.1  1 
AIR150.1  1 
AIR150.2  1 
AIR241.1  1 
AIR526.1  1 
AIR528.1  1 
AIR540.1  1 
AIR550.1  1 
AIR550.2  1 
AIR561.1  1 
AIR561.2  1 
AIR564.1  1 
AIR567.1  1 
AIR567.2  1 
AIR602.1  1 
AIR800.1  1 
AIR800.2  1 
AIR800.3  1 
AIR810.1  1 
AIR810.2  1 
AIR830.1  1 
AIR830.2  1 
AIR830.3  1 
AIR830.4  1 
AIR840.1  1 
AIR840.2  1 
AIR840.3  1 
AIR840.4  1 
AIR911.1  1 
AIR911.2  1 
AIR920.1  1 
ANT211.1  1 
ANT211.2  1 
ANT211.3  1 
BSEOOl.l  1 
BSE001.2  1 
BSE002.1  1 
BSE002.2  1 
BSE003.1  1 
BSE003.2  1 
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BSE020.1  1 
BSE030.1  1 
BSE050.1  1 
BSE050.2  1 
BSE200.1  1 
BSE200.2  1 
BSE200.3  1 
BSE200.4  1 
BSE200.5  1 
BSE201.1  1 
BSE201.2  1 
BSE251.1  1 
BSE260.1  1 
BSE260.2  1 
BSE270.1  1 
BSE400.1  1 
BSE800.1  1 
GRD095.1  1 
GRD095.2  1 
GRD095.3  1 
GRD100.1  1 
GRD300.1  1 
GRD302.1  1 
GRD309.1  1 
GRD871.1  1 
GRD871.2  1 
ISROOO.l  1 
ISR000.2  1 
ISR000.3  1 
ISR000.4  1 
ISROOl.l  1 
ISR001.2  1 
ISR001.3  1 
ISR004.1  1 
ISR005.1  1 
ISR115.1  1 
ISR115.2  1 
ISR250.1  1 
ISR250.2  1 
ISR251.1  1 
ISR251.2  1 
ISR251.3  1 
ISR251.4  1 
ISR360.1  1 
ISR365.1  1 
ISR400.1  1 
ISR500.1  1 
ISR500.2  1 
ISR500.3  1 
LOG700.1  1 
LOG700.2  1 
LOG700.3  1 
PLAOlO.l  1 
P LAO 10 . 2  1 
PLA443.1  1 
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PLA557.1  1 
REC011.1  1 
REC011.2  1 
REC011.3  1 
REC012.1  1 
REC012.2  1 
REC012.3  1 
REC012.4  1 
REC012.5  1 
REC012.6  1 
REC013.1  1 
REC013.2  1 
REC013.3  1 
REC013.4  1 
REC013.5  1 
REC013.6  1 
REC014.1  1 
REC014.2  1 
REC014.3  1 
REC014.4  1 
REC014.5  1 
REC014.6  1 
REC014.7  1 
REC014.8  1 
REC014.9  1 
REC015.1  1 
REC015.2  1 
REC015.3  1 
REC015.4  1 
REC015.5  1 
REC015.6  1 
REC015.7  1 
REC015.8  1 
REC016.1  1 
REC016.2  1 
REC016.3  1 
REC016.4  1 
REC016.5  1 
REC016.6  1 
REC016.7  1 
REC016.8  1 
REC016.9  1 
REC016 . 10  1 
REC016.il  1 
REC017.1  1 
REC017.2  1 
REC017.3  1 
REC017.4  1 
REC017.5  1 
REC017.6  1 
REC017.7  1 
REC017.8  1 
REC017.9  1 
REC017 . 10  1 
REC018.1  1 
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REC018.2  1 

REC018.3  1 

REC018.4  1 

REC310.1  1 

REC310.2  1 

REC320.1  1 

REC320.2  1 

REC320.3  1 

REC330.1  1 

REC330.2  1 

REC330.3  1 

REC330.4  1 

REC330.5  1 

REC330.6  1 

REC330.7  1 

REC330.8  1 

REC341.1  1 

REC341.2  1 

REC341.3  1 

REC341.4  1 

REC900.1  1 

REC900.2  1 

UTL104.1  1 

UTL305.1  1 

UTL520.1  1 

UTL520.2  1 

UTL550.1  1 

UTL550.2  1 

UTL750.1  1 

UTL920.1  1 

ISR000.5  1 

ADF140.4  1 

ADF440.4  1 

AIR830.5  1 

AIR830.6  1 

ISR251.5  1 

ISR251.6  1 

ISR251.7  1 

LOG700.4  1 

END. RN. STREAMS 

END . RANDOM . NUMBER . STREAM . DATA 
END. CONTROL 
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Appendix  E:  Example  SIMSCRIPT  IL5  Code  AIR070.SIM  (Original) 


ROUTINE  AIR. 070 .ADJUDICATE . DTBM 

' ' $ I d :  air070.sim,v  2.10  2000/09/20  20:12:38  steveh  Exp  $ 


GIVEN 

.DTBM. PLATFORM, 

. DTBM. WPN. TYPE, 

.  TBU, 

.TBM.FLT, 

. DETECT . SECONDS . DELAY 


'  PTR  TO  DTBM  PLATFORM  FIRING  AT  TBM 
'TYPE  OF  SHOOTER  ENGAGING  TBM 
'INDEX  OF  TBM. UNIT  LAUNCHING  TBM 
'POINTER  TO  TBM  FLIGHT  BEING  LAUNCHED 
'BETWEEN  TBM  LAUNCH  AND  DTBM  RESPONSE 


'YIELDING 

'  NONE 

'  PURPOSE 

'  ADJUDICATE 

A  DTBM  VS 

.  TBM  AIR-TO-AIR  ENGAGEMENT 

'  PROCESSING 

'CALLED  BY 

'  AIR. 060. BOOST. PHASE. 

INTERCEPT . MANAGER 

'CHANGE  LOG 

'  DATE 

AUTHOR 

CHANGE  SUMMARY 

'  1-11-96 

MORGAN 

INITIAL  IMPLEMENTATION 

'  1-23-96 

EMERT 

REPLACED  . TBT  WITH  . TTT 

'  2-6-96 

MORGAN 

SET  FL . ENGAGE . FLAG 

'  2-9-96 

MORGAN 

DISPLAY  ENGAGEMENTS  ALONG  TBM  PATH 

'  6-12-96 

MORGAN 

MAKE  LASER  SHOT  TIME  A  DISTRIBUTION 

'  8-09-99 

HADLEY 

FLIGHT  GROUP  MISSION  STATS  MAY  HAVE 

BEEN  CREATED  EARLIER 

'  08-16-99 

LENNOX 

CALL  GRA. 730,  GRA.740  FOR  GRAPHER2K 

'  2-11-00 

COSGROVE 

ADD  SBL 

O  I 
o  | 

1  1 
O  I 
\ — 1  1 

1  1 
r-  1 
o  | 

HADLEY 

UP  FLIGHT  GROUP  ID  IN  TRANS  TO  I  7 

NORMALLY  MODE  IS  INTEGER 

DEFINE  .DETECT. SECONDS. DELAY,  . PK,  . SHOT . DURATION . SEC,  . SUM . SECONDS , 
. TOTAL . ELAPSED . SECONDS 
AS  DOUBLE  VARIABLES 

IF  .DTBM. WPN. TYPE  =  .. SPACE . BASED . LASER 
LET  .DTBM. SAT  =  . DTBM . PLATFORM 
ELSE 
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LET  .DTBM.FG 


.DTBM. PLATFORM 


LET  .DTBM.FLT  =  F . FG . FLIGHT . SET (. DTBM . FG) 

IF  FL. MISSION ( .DTBM.FLT)  <>  ..DTBM 
TRACE 
STOP 

OTHERWISE 

ALWAYS 

LET  .TBM.FG  =  FL . FLT . GP ( . TBM. FLT) 

LET  .TBM. LAUNCH. X  =  TBU . X . COORD (. TBU) 

LET  .TBM. LAUNCH. Y  =  TBU . Y . COORD (. TBU) 

LET  .TBM.TAC  =  TBU . TYPE . AC (. TBU) 

LET  . TTT  =  TA. TBM. TARGET. TYPE ( .TBM.TAC) 

LET  .TBM.TGT  =  FG . TARGET (. TBM . FG) 

LET  .TBM.TGT.X  =  TG . X . COORD (. TBM . TGT ) 

LET  .TBM. TGT. Y  =  TG . Y . COORD (. TBM . TGT ) 

IF  . DTBM. WPN. TYPE  =  .. SPACE . BASED . LASER 

LET  . TSW  =  SAT. TYPE. SPACE. WPN (.DTBM. SAT) 

LET  . DWP  =  TSW. DTBM. WEAPON ( .TSW) 

LET  .PLATFORM. ID  =  SAT . ID (. DTBM. SAT) 

ELSE 

FOR  EVERY  .AM  IN  FL . MUNITION . SET (. DTBM . FLT ) 

WITH  (AM. FUNCTION ( .AM)  =  ..LASER 

OR  AM. FUNCTION ( .AM)  =  . . KKV) 

AND  AM. RNDS .REMAINING ( .AM)  >  0 
FIND  THE  FIRST  CASE 
IF  NONE 
TRACE 
STOP 

OTHERWISE 

LET  .TAM  =  AM. TYPE (.AM) 

LET  .DWP  =  TAM. DTBM. WEAPON ( .TAM) 

LET  .PLATFORM. ID  =  FG . ID ( . DTBM . FG) 

ALWAYS 

' 'determine  TBM  flight  phase  (and  pK)  at  which  to  adjudicate 
IF  .DTBM. WPN. TYPE  =  .. AIRBORNE . LASER 

OR  .DTBM. WPN. TYPE  =  .. SPACE . BASED . LASER 

CALL  UTL .720. CALCULATE . USER . FUNCTION 
GIVEN 

DWP . SHOT . DURATION . FN ( . DWP)  , 

RN. STREAM (. .AIR070.1) 

YIELDING 

. SHOT . DURATION . SEC 

LET  .SHOT. DURATION. SEC  =  MAX . F (. SHOT . DURATION . SEC,  0.0) 
LET  .TOTAL. ELAPSED. SECONDS 

=  .DETECT. SECONDS. DELAY  +  . SHOT . DURATION . SEC 
LET  . INTERCEPT. FLAG  =  ..YES 

ELSE 

IF  .DTBM. WPN. TYPE  <>  ..KKV 
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TRACE 

STOP 

OTHERWISE 

LET  .DTBM.X  =  FG . X . COORD (. DTBM . FG) 

LET  . DTBM. Y  =  FG . Y . COORD (. DTBM . FG) 

LET  . DTBM. Z  =  PTH . ALTITUDE ( FG . CURR . PATH (. DTBM . FG) ) 

CALL  AIR. 071 . COMPUTE . KKV. INTERCEPT .TIME 
GIVEN 

. TTT,  ''INDEX  OF  TBM . TARGET . TYPE 

.TBM. LAUNCH. X, 

.TBM. LAUNCH. Y, 

.TBM.TGT.X, 

.TBM.TGT.Y, 

. DWP,  ''INDEX  OF  DTBM. WEAPON 

.DTBM.X, 

. DTBM. Y, 

. DTBM. Z, 

. DETECT . SECONDS . DELAY  ' ' FROM  TBM  LAUNCH  TO  KKV  LAUNCH 
YIELDING 

. INTERCEPT. FLAG,  ''..YES  OR  ..NO  SUCCESSFUL  INTERCEPT 

.TOTAL. ELAPSED. SECONDS  ''FROM  TBM  LAUNCH  TO  KKV  INTERCEPT 

ALWAYS 

IF  . INTERCEPT. FLAG  =  ..YES 

FOR  EVERY  TBM . FLIGHT . PHASE  CALLED  . TFP 
DO 

ADD  TPT. DURATION. SEC ( .TFP,  .TTT)  TO  .SUM. SECONDS 
IF  .TOTAL. ELAPSED. SECONDS  <=  .SUM. SECONDS 
LEAVE 
OTHERWISE 
LOOP 

IF  .TFP  >  N. TBM. FLIGHT. PHASE 
LET  . PK  =  0.0 
ELSE 

LET  .PK  =  DPT . PK ( . DWP,  .TFP,  .TTT) 

ALWAYS 

ELSE  ' 'KKV  failed  to  intercept 
LET  .PK  =  0.0 
ALWAYS 


' 'determine  result  of  shot 
IF  .PK  >  0.0 

AND  RANDOM. F(RN. STREAM (. .AIR070. 2) )  <=  . PK 
LET  .NUM. KILLED  =  1 
ELSE 

LET  .NUM. KILLED  =  0 
ALWAYS 


IF  (DT . PRINT . FLAG ( . . AIRF)  >=  . . LV3  AND 

DT . START . TIME ( . . AIRF)  <=  TIME.V  <=  DT . STOP . TIME (. .AIRF) ) 
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OR  (DT. PRINT. FLAG (. .BPIPP)  >  . . LVO  AND 

DT. START. TIME (. .BPIPP)  <=  TIME.V  <=  DT . STOP . TIME (.. BPIPP) ) 

' 'Result  of  DTBM  engagement  with  a  TBM 
WRITE  TIME.V, 

TBU. ID ( . TBU)  , 

FG. ID (FL.FLT.GP ( .TBM.FLT)  )  , 

TA. ID ( .TBM.TAC) , 

. PLATFORM. ID, 

DWP. ID ( . DWP) , 

. TOTAL . ELAPSED . SECONDS , 

. INTERCEPT. FLAG, 

.PK, 

.NUM. KILLED 

AS  D ( 8 , 4 ) ,  "  AIR070013  ",  I  5 ,  S  1 ,  I  7 ,  S  1 ,  I  5 ,  S  1 ,  I  7 ,  S  1 , 

S  1,  I  5,  S  1,  D  (6, 1)  ,  S  1,  I  1,  S  1,  D  (5, 3)  ,  S  1,  I  1,  / 

ALWAYS 

' ' expend  weapon 

IF  . DTBM. WPN. TYPE  =  .. SPACE . BASED . LASER 

SUBTRACT  1  FROM  SAT . SPACE . WPN . SHOTS . REMAINING (. DTBM . SAT ) 

IF  SAT. SPACE. WPN. SHOTS. REMAINING ( . DTBM. SAT)  <  0 
TRACE 
STOP 

OTHERWISE 

ELSE 

IF  DT . PRINT . FLAG ( . . DB . EXEC)  >=  . . LV1  AND 

DT . START . TIME ( . . DB . EXEC)  <=  TIME.V  <=  DT . STOP . TIME (.. DB . EXEC) 
CALL  DBS . 2 2 0 . BEGIN . AIR . TO . AIR . ENGAGEMENT 
GIVEN 

. DTBM. X, 

. DTBM. Y 

ALWAYS 

SUBTRACT  1  FROM  AM . RNDS . REMAINING (. AM) 

LET  . TMI  =  TAM. INSTANCE ( .TAM) 

LET  .EXPENDED  =  TAM . NUMBER (. TAM) 

IF  TMI .CRITICAL. RESOURCE (TAM. INSTANCE ( .TAM) )  >  0 

SUBTRACT  .EXPENDED  FROM  TCR . LOADED ( TMI . CRITICAL . RESOURCE (. TMI ) ) 
ADD  .EXPENDED  TO  TCR. EXPENDED (TMI . CRITICAL . RESOURCE (. TMI ) ) 
ALWAYS 

IF  DT . PRINT . FLAG ( . . DB . EXEC)  >=  . . LV1  AND 

DT . START . TIME ( . . DB . EXEC)  <=  TIME.V  <=  DT . STOP . TIME (.. DB . EXEC) 
CALL  DBS. 222 .MUNITION. EXPEND. AIR. TO. AIR 
GIVEN 

. DTBM. FLT, 

. DTBM. FG, 

.TMI, 

.EXPENDED 

ALWAYS 

IF  DT. PRINT. FLAG (. .GRAPHER2K)  >=  . . LV1  AND 
DT. START. TIME (. .GRAPHER2K)  <=  TIME.V  <= 
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DT . STOP .TIME ( . . GRAPHER2K) 


CALL  GRA. 730. AIR. MUNITION. 
GIVEN 

. .AA.MUNT. LAUNCHED, 
.DTBM.FLT, 

.TAM, 

1, 

0, 

0, 

0, 

0 

ALWAYS 


EXPENDITURE 


'expenditure  type 


'  #  of  munt  sticks 
'not  used  here 
'not  used  here 
'  not  used  here 
'not  used  here 


IF  DT. PRINT. FLAG (. .GRAPHER2K)  >=  . . LV1  AND 
DT. START. TIME (. .GRAPHER2K)  <=  TIME.V  <= 
DT . STOP .TIME ( . . GRAPHER2K) 


IF  .NUM. KILLED  >  0 

CALL  GRA. 740. AIR. TO. AIR. KILLS 
GIVEN 

.TBM.FLT,  ''victim  flight 

.DTBM.FLT,  ''killer  flight 

.NUM. KILLED  ''losses  to  victim  flight 

ALWAYS 

ALWAYS 


' 'set  up  DTBM  flight  group  statistics  for  air  reports 
IF  FG. MSN. STAT ( .DTBM. FG)  <>  0 
TRACE 
STOP 

OTHERWISE 


CREATE  MISSION. STATISTIC  CALLED  .MS 

LET  MS. FLIGHT. GROUP. ID (.MS)  =  FG . ID ( . DTBM . FG) 

LET  MS. TIME (.MS)  =  TIME.V 

LET  MS. TARGET. TYPE ( .MS)  =  .. INTERCEPT . POINT 
LET  MS. TARGET. ID (.MS)  =  FG . ID ( . TBM . FG) 

LET  MS. TARGET. NAME ( .MS)  =  "" 

LET  MS .KILLS ( .MS)  =  .NUM. KILLED 
LET  FG.MSN. STAT ( .DTBM. FG)  =  .MS 

CREATE  FLIGHT. STAT  CALLED  . FS 

LET  FLS. FLIGHT. ID(.FS)  =  FL . ID ( . DTBM. FLT) 

LET  FLS. TYPE. AC (.FS)  =  FL . TYPE . AC (. DTBM . FLT ) 

LET  FLS. NUM. AC (.FS)  =  FL . NUM . AC ( . DTBM . FLT ) 

LET  FLS. LOSSES ( .FS)  =  0 

FILE  .FS  IN  MS. FLIGHT. STAT. SET ( .MS) 

CREATE  TARGET. COMPONENT. STAT  CALLED  . TCS 

LET  TCS. TYPE. COMPONENT ( .TCS)  =  FL . TYPE . AC ( . TBM . FLT ) 

LET  TCS. COMPONENT. ID (.TCS)  =  FL . ID ( . TBM. FLT) 

LET  TCS . QTY . PCT . KILLED ( . TCS )  =  .NUM. KILLED 
FILE  .TCS  IN  MS. TARGET. COMPONENT. SET ( .MS) 
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CREATE  AIR. MUNITION. STAT  CALLED  .AMS 
LET  AMS. TYPE ( .AMS)  =  .TAM 
LET  AMS. LAUNCHES ( .AMS)  =  1 

LET  AMS. TGT. TYPE. COMPONENT ( .AMS)  =  ..AIRCRAFT 

LET  AMS. TGT. COMPONENT. ID (.AMS)  =  FL . TYPE . AC ( . TBM . FLT ) 

FILE  .AMS  IN  FLS. MUNITION. SET ( .FS) 

IF  FG. MSN. STAT ( .TBM. FG)  =  0 

CREATE  A  MISSION. STATISTIC  CALLED  .MS 
LET  FG. MSN. STAT ( .TBM. FG)  =  .MS 
ELSE 

LET  .MS  =  FG.MSN. STAT ( .TBM. FG) 

ALWAYS 

LET  MS. FLIGHT. GROUP. ID (.MS)  =  FG . ID ( . TBM . FG) 

LET  MS. TIME (.MS)  =  TIME.V 

LET  MS. TARGET. TYPE ( .MS)  =  .. INTERCEPT . POINT 
LET  MS. TARGET. ID (.MS)  =  FG . ID ( . DTBM . FG) 

LET  MS. TARGET. NAME ( .MS)  =  "" 

LET  MS .KILLS ( .MS)  =  0 

CREATE  FLIGHT. STAT  CALLED  . FS 

LET  FLS. FLIGHT. ID(.FS)  =  FL . ID ( . TBM. FLT) 

LET  FLS. TYPE. AC ( .FS)  =  FL . TYPE . AC ( . TBM . FLT ) 

LET  FLS.NUM.AC ( .FS)  =  FL . NUM . AC ( . TBM . FLT ) 

LET  FLS. LOSSES ( .FS)  =  .NUM. KILLED 
FILE  .FS  IN  MS. FLIGHT. STAT. SET ( .MS) 

CREATE  TARGET. COMPONENT. STAT  CALLED  . TCS 

LET  TCS. TYPE. COMPONENT ( .TCS)  =  FL . TYPE . AC (. DTBM . FLT ) 

LET  TCS. COMPONENT. ID (.TCS)  =  FL . ID ( . DTBM. FLT) 

LET  TCS . QTY . PCT . KILLED ( . TCS )  =  0 

FILE  .TCS  IN  MS. TARGET. COMPONENT. SET ( .MS) 

IF  DT. PRINT. FLAG ( . .AA.ENC)  >=  . . LV1  AND 

DT. START. TIME ( . .AA.ENC)  <=  TIME.V  <=  DT . STOP . TIME (.. AA . ENC ) 
CALL  UTL . 503 . LOCATE . SECTOR. FLOT . X 
GIVEN 

.TBM. LAUNCH. X, 

.TBM. LAUNCH. Y, 

. .BLUE 
YIELDING 
. FLOT . X 

LET  . BLUE. DIST. TO. FLOT  =  . TBM . LAUNCH . X  -  . FLOT . X 

CALL  UTL . 503 . LOCATE . SECTOR. FLOT . X 
GIVEN 

.TBM. LAUNCH. X, 

.TBM. LAUNCH. Y, 

.  .RED 
YIELDING 
. FLOT . X 

LET  .RED. DIST. TO. FLOT  =  . FLOT . X  -  . TBM . LAUNCH . X 
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' 'Number  of  flights  in  the  air  engagement  and  the  distance 
' 'from  the  sector  FLOT  for  each  side 
WRITE  TIME . V, 

1 ,  ' ' blue  flight 

. BLUE. DIST. TO. FLOT, 

1 ,  ' ' red  flight 

.RED. DIST. TO. FLOT 

AS  D ( 8 , 4 ) , "  AIR070091  ",I  3,S  1,1  9,S  1,1  3,S  1,1  9,  / 

' 'Describes  a  DTBM  flight  in  an  air  engagement. 

WRITE  TIME . V, 

TA. USED. INDEX(FL. TYPE. AC ( . DTBM. FLT)  )  , 

FL. MISS ION ( .DTBM. FLT)  , 

FL . NUM . AC ( . DTBM . FLT ) , 

FG. STATUS (FL.FLT.GP ( .DTBM. FLT) ) 

AS  D ( 8 , 4 ) , "  AIR070101  ",I  5,S  1,1  5,S  1,1  5,S  1,1  5,  / 

' 'Describes  a  TBM  flight  in  an  air  engagement. 

WRITE  TIME . V, 

TA. USED. INDEX(FL. TYPE. AC ( . TBM. FLT)  )  , 

FL. MISS ION ( .TBM. FLT)  , 

FL . NUM . AC ( . TBM . FLT ) , 

FG. STATUS (FL.FLT.GP ( .TBM. FLT) ) 

AS  D ( 8 , 4 ) , "  AIR070101  ",I  5,S  1,1  5,S  1,1  5,S  1,1  5,  / 

ALWAYS 

IF  DT. PRINT. FLAG( . .AA. KILLS)  >=  . . LV1  AND 

DT. START. TIME (. .AA. KILLS)  <=  TIME.V  <=  DT . STOP . TIME (.. AA . KILLS ) 
' 'Air-to-Air  kills  for  post  processor 
WRITE  TIME.V, 

TA. USED. INDEX(FL. TYPE. AC ( . DTBM. FLT)  )  , 

FL. MISS ION ( .DTBM. FLT)  , 

FL . NUM . AC ( . DTBM . FLT ) , 

TA. USED. INDEX(FL. TYPE. AC ( . TBM. FLT) ) , 

FL. MISS ION ( .TBM. FLT) , 

FL . NUM . AC ( . TBM . FLT ) , 

0  ' ' num  lost  by  DTBM 

AS  D ( 8 , 4 ) ,  "  AIR070081",  S  1 ,  I  5 ,  S  1 ,  I  5 ,  S  1 , 

I  5,  S  1,  I  5,  S  1,  I  5,  S  1,  I  5,  S  1,  I  5,  / 

' 'Air-to-Air  kills  for  post  processor 
WRITE  TIME.V, 

TA. USED. INDEX(FL. TYPE. AC ( . TBM. FLT) ) , 

FL. MISS ION ( .TBM. FLT)  , 

FL . NUM . AC ( . TBM . FLT ) , 

TA. USED. INDEX(FL. TYPE. AC ( . DTBM. FLT) ) , 

FL. MISS ION ( .DTBM. FLT)  , 

FL . NUM . AC ( . DTBM . FLT ) , 

.NUM. KILLED 

AS  D ( 8 , 4 ) ,  "  AIR070081",  S  1 ,  I  5 ,  S  1 ,  I  5 ,  S  1 , 

I  5,  S  1,  I  5,  S  1,  I  5,  S  1,  I  5,  S  1,  I  5,  / 

ALWAYS 

' ' report  results 
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CALL  UTL. 270. MISSION. STAT. REPORT 
GIVEN 

.DTBM.FG, 

. . DTBM 

CALL  UTL. 270. MISSION. STAT. REPORT 
GIVEN 

.TBM.FG, 

. .ALL. FLIGHTS 

ALWAYS 


IF  .NUM. KILLED  =  1 

IF  . DTBM. WPN. TYPE  <>  .. SPACE . BASED . LASER 

IF  DT. PRINT. FLAG ( . .DB.EXEC)  >=  . . LV1  AND 
DT. START. TIME ( . .DB.EXEC)  <=  TIME.V  <= 

DT . STOP .TIME ( . . DB . EXEC) 

CALL  DBS. 221. FLT. LOSS. AIR. TO. AIR 
GIVEN 

.TBM.FLT, 

.TBM.FG, 

1 ,  ' ' loss 

..YES,  ''before  target 

TA. ID ( .TBM.TAC) , 

FL. MISS ION ( .TBM.FLT) 

ALWAYS 

IF  (POST. PROCESS. GRAPHICS  =  ..MAP  OR 

POST. PROCESS. GRAPHICS  =  ..BOTH)  AND 
DT. PRINT. FLAG (. .GRAA)  GE  . . LV2  AND 

DT. START. TIME (. .GRAA)  LE  TIME.V  LE  DT . STOP . TIME (.. GRAA) 

IF  FG. SIDE ( .DTBM.FG)  =  ..BLUE 
LET  . BLUE . FG  =  .DTBM.FG 
LET  . RED . FG  =  .TBM.FG 
ELSE 

LET  . BLUE . FG  =  .TBM.FG 
LET  . RED . FG  =  .DTBM.FG 
ALWAYS 

' ' FOR  NOW 


LET 

. ENGAGE. X  = 

.5  *  ( .TBM. LAUNCH. X  + 

.TBM.TGT.X) 

LET 

. ENGAGE. Y  = 

.5  *  ( .TBM. LAUNCH. Y  + 

.TBM.TGT.Y) 

LET 

. SAVE. TBM. X 

=  FG. X. COORD ( .TBM.FG) 

LET 

. SAVE. TBM. Y 

=  FG. Y. COORD ( .TBM.FG) 

LET  FG. X. COORD ( .TBM.FG)  =  . ENGAGE . X 
LET  FG. Y. COORD ( .TBM.FG)  =  . ENGAGE . Y 

IF  DT. PRINT. FLAG (. .GRAA)  >=  . . LV3 

LET  FG.CURR. PATH (.TBM.FG)  =  F . FG . PATH . SET (. TBM . FG) 

CALL  GRA .600. PRINT .MISSION. MOVE . EVENT 
GIVEN 

.TBM.FG, 
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.TBM.FLT 


LET  FG . CURR. PATH ( . TBM. FG)  =  0 
ALWAYS 

LET  FL. NUM. AC. LOST. LAST. ATTACK ( .TBM.FLT)  =  .NUM. KILLED 
LET  FL. ENGAGE. FLAG ( .TBM.FLT)  =  ..YES 
LET  FL. ENGAGE. FLAG ( .DTBM.FLT)  =  ..YES 

''TEMPORARILY  MOVE  DTBM  FLIGHT  GROUP  SO  GRAPHICS  WILL  SHOW 
' '  ENGAGEMENT  ALONG  TBM  FLIGHT  PATH 
LET  .SAVE. DTBM. X  =  FG . X . COORD (. DTBM . FG) 

LET  .SAVE. DTBM. Y  =  FG . Y . COORD (. DTBM . FG) 

LET  FG. X. COORD ( .DTBM. FG)  =  . ENGAGE . X 
LET  FG. Y. COORD ( .DTBM. FG)  =  . ENGAGE . Y 

CALL  GRA. 540. AIR. TO. AIR. LOSS 
GIVING 

.BLUE . FG, 

. RED . FG 

LET  FG. X. COORD ( .DTBM. FG)  =  .SAVE. DTBM. X 
LET  FG. Y. COORD ( .DTBM. FG)  =  .SAVE. DTBM. Y 
LET  FG. X. COORD ( .TBM. FG)  =  .SAVE. TBM. X 
LET  FG. Y. COORD ( .TBM. FG)  =  .SAVE. TBM. Y 

LET  FL. NUM. AC. LOST. LAST. ATTACK ( .TBM.FLT)  =  0 
LET  FL. ENGAGE. FLAG ( .TBM.FLT)  =  ..NO 
LET  FL. ENGAGE. FLAG ( .DTBM.FLT)  =  ..NO 
ALWAYS 
ALWAYS 

' 'clean  up  tbm  flight  &  flight  group 
CALL  UTL. 090. DESTROY. AIRCRAFT 
GIVEN 

.TBM.FLT, 

.NUM. KILLED, 

. .AIRCRAFT 


CALL  AIR. 510. DECISION. POINT 
GIVEN 

. TBM. FG, 

..YES  ''aircraft  lost 

YIELDING 

.CONTINUE 

IF  .CONTINUE  <>  ..NO 
TRACE 
STOP 

OTHERWISE 

FOR  EVERY  . EVENT4  IN  EV . S ( I . AIR . 0 04 . MOVE . FLT . GP) 
WITH  MFG.FLT.GP ( .EVENT4)  =  . TBM. FG 
FIND  THE  FIRST  CASE 
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IF  NONE 
TRACE 
STOP 

OTHERWISE 

CANCEL  AIR. 004. MOVE. FLT.GP  CALLED  . EVENT4 
DESTROY  AIR. 004. MOVE. FLT.GP  CALLED  . EVENT4 
ALWAYS 


IF  . DTBM . WPN . TYPE  <>  .. SPACE . BASED . LASER 

IF  DT. PRINT. FLAG ( . .DB.EXEC)  >=  . . LV1  AND 

DT. START. TIME ( . .DB.EXEC)  <=  TIME.V  <=  DT . STOP . TIME (.. DB . EXEC) 
CALL  DBS . 2 2 9 . END . AIR . TO . AIR . ENGAGEMENT 
ALWAYS 

' 'continuation  decision  of  no  will  cause  DTBM  to  leave  station 
CALL  AIR. 510. DECISION. POINT 
GIVEN 

.DTBM.FG, 

. .NO  ' 'aircraft  lost 

YIELDING 

.CONTINUE 

ALWAYS 

EXITROUTINE 

ENDROUTINE 
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Appendix  F:  Example  SIMSCRIPT  II.5  Code  AIR070.SIM  (Modified) 


ROUTINE  AIR. 070 .ADJUDICATE . DTBM 

' ' $ I d :  air070.sim,v  2.10  2000/09/20  20:12:38  steveh  Exp  $ 


GIVEN 

.DTBM. PLATFORM, 

. DTBM. WPN. TYPE, 

.  TBU, 

.TBM.FLT, 

. DETECT . SECONDS . DELAY 


'  PTR  TO  DTBM  PLATFORM  FIRING  AT  TBM 
'TYPE  OF  SHOOTER  ENGAGING  TBM 
'INDEX  OF  TBM. UNIT  LAUNCHING  TBM 
'POINTER  TO  TBM  FLIGHT  BEING  LAUNCHED 
'BETWEEN  TBM  LAUNCH  AND  DTBM  RESPONSE 


'YIELDING 

'  NONE 

'  PURPOSE 

'  ADJUDICATE 

A  DTBM  VS 

.  TBM  AIR-TO-AIR  ENGAGEMENT 

'  PROCESSING 

'CALLED  BY 

'  AIR. 060. BOOST. PHASE. 

INTERCEPT . MANAGER 

'CHANGE  LOG 

'  DATE 

AUTHOR 

CHANGE  SUMMARY 

'  1-11-96 

MORGAN 

INITIAL  IMPLEMENTATION 

'  1-23-96 

EMERT 

REPLACED  . TBT  WITH  . TTT 

'  2-6-96 

MORGAN 

SET  FL . ENGAGE . FLAG 

'  2-9-96 

MORGAN 

DISPLAY  ENGAGEMENTS  ALONG  TBM  PATH 

'  6-12-96 

MORGAN 

MAKE  LASER  SHOT  TIME  A  DISTRIBUTION 

'  8-09-99 

HADLEY 

FLIGHT  GROUP  MISSION  STATS  MAY  HAVE 

BEEN  CREATED  EARLIER 

'  08-16-99 

LENNOX 

CALL  GRA. 730,  GRA.740  FOR  GRAPHER2K 

'  2-11-00 

COSGROVE 

ADD  SBL 

O  I 
o  | 

1  1 
O  I 
\ — 1  1 

1  1 
r-  1 
o  | 

HADLEY 

UP  FLIGHT  GROUP  ID  IN  TRANS  TO  I  7 

NORMALLY  MODE  IS  INTEGER 

DEFINE  .DETECT. SECONDS. DELAY,  . PK,  . SHOT . DURATION . SEC,  . SUM . SECONDS , 
. TOTAL . ELAPSED . SECONDS 
AS  DOUBLE  VARIABLES 

IF  .DTBM. WPN. TYPE  =  .. SPACE . BASED . LASER 
LET  .DTBM. SAT  =  . DTBM . PLATFORM 
ELSE 
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LET  .DTBM.FG 


.DTBM. PLATFORM 


LET  .DTBM.FLT  =  F . FG . FLIGHT . SET (. DTBM . FG) 

IF  FL. MISSION ( .DTBM.FLT)  <>  ..DTBM 
TRACE 
STOP 

OTHERWISE 

ALWAYS 

LET  .TBM.FG  =  FL . FLT . GP ( . TBM. FLT) 

LET  .TBM. LAUNCH. X  =  TBU . X . COORD (. TBU) 

LET  .TBM. LAUNCH. Y  =  TBU . Y . COORD (. TBU) 

LET  .TBM.TAC  =  TBU . TYPE . AC (. TBU) 

LET  . TTT  =  TA. TBM. TARGET. TYPE ( .TBM.TAC) 

LET  .TBM.TGT  =  FG . TARGET (. TBM . FG) 

LET  .TBM.TGT.X  =  TG . X . COORD (. TBM . TGT ) 

LET  .TBM. TGT. Y  =  TG . Y . COORD (. TBM . TGT ) 

IF  . DTBM. WPN. TYPE  =  .. SPACE . BASED . LASER 

LET  . TSW  =  SAT. TYPE. SPACE. WPN (.DTBM. SAT) 

LET  . DWP  =  TSW. DTBM. WEAPON ( .TSW) 

LET  .PLATFORM. ID  =  SAT . ID (. DTBM. SAT) 

ELSE 

FOR  EVERY  .AM  IN  FL . MUNITION . SET (. DTBM . FLT ) 

WITH  (AM. FUNCTION ( .AM)  =  ..LASER 

OR  AM. FUNCTION ( .AM)  =  . . KKV) 

AND  AM. RNDS .REMAINING ( .AM)  >  0 
FIND  THE  FIRST  CASE 
IF  NONE 
TRACE 
STOP 

OTHERWISE 

LET  .TAM  =  AM. TYPE (.AM) 

LET  .DWP  =  TAM. DTBM. WEAPON ( .TAM) 

LET  .PLATFORM. ID  =  FG . ID ( . DTBM . FG) 

ALWAYS 

' 'determine  TBM  flight  phase  (and  pK)  at  which  to  adjudicate 
IF  .DTBM. WPN. TYPE  =  .. AIRBORNE . LASER 

OR  .DTBM. WPN. TYPE  =  .. SPACE . BASED . LASER 

CALL  UTL .720. CALCULATE . USER . FUNCTION 
GIVEN 

DWP . SHOT . DURATION . FN ( . DWP)  , 

RN. STREAM (. .AIR070.1) 

YIELDING 

. SHOT . DURATION . SEC 


'  '  *  Earl  stuff  * 


EARL. RESPONSE  =  . SHOT . DURATION . SEC 

EB.P1  =  UF.PARM1 (DWP. SHOT. DURATION. FN ( . DWP) ) 

EB.P2  =  UF.PARM2 (DWP. SHOT. DURATION. FN ( . DWP) ) 
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EB.  P3 


0.0 


CALL  EARL. STORE 
GIVEN 

. . AIR070 . 1, 

UF. FUNCTION (DWP. SHOT. DURATION. FN(.DWP) ) , 

EARL. RESPONSE, 

EB. PI, 

EB. P2, 

EB.  P3 

'  '  *  END  STUFF 

LET  .SHOT. DURATION. SEC  =  MAX . F (. SHOT . DURATION . SEC,  0.0) 

LET  .TOTAL. ELAPSED. SECONDS 

=  .DETECT. SECONDS. DELAY  +  . SHOT . DURATION . SEC 
LET  . INTERCEPT. FLAG  =  ..YES 

ELSE 

IF  . DTBM . WPN . TYPE  <>  . . KKV 
TRACE 
STOP 

OTHERWISE 

LET  . DTBM. X  =  FG . X . COORD (. DTBM . FG) 

LET  . DTBM. Y  =  FG . Y . COORD (. DTBM . FG) 

LET  . DTBM. Z  =  PTH . ALTITUDE ( FG . CURR . PATH (. DTBM . FG) ) 

CALL  AIR. 071 . COMPUTE . KKV. INTERCEPT .TIME 
GIVEN 

. TTT,  ''INDEX  OF  TBM . TARGET . TYPE 

.TBM. LAUNCH. X, 

.TBM. LAUNCH. Y, 

.TBM.TGT.X, 

.TBM.TGT.Y, 

.DWP,  ''INDEX  OF  DTBM. WEAPON 

. DTBM. X, 

. DTBM. Y, 

. DTBM. Z, 

. DETECT . SECONDS . DELAY  ' ' FROM  TBM  LAUNCH  TO  KKV  LAUNCH 

YIELDING 

. INTERCEPT. FLAG,  ''..YES  OR  ..NO  SUCCESSFUL  INTERCEPT 

.TOTAL. ELAPSED. SECONDS  ''FROM  TBM  LAUNCH  TO  KKV  INTERCEPT 

ALWAYS 

IF  . INTERCEPT. FLAG  =  ..YES 

FOR  EVERY  TBM . FLIGHT . PHASE  CALLED  . TFP 
DO 

ADD  TPT. DURATION. SEC ( .TFP,  .TTT)  TO  .SUM. SECONDS 
IF  .TOTAL. ELAPSED. SECONDS  <=  .SUM. SECONDS 
LEAVE 
OTHERWISE 
LOOP 

IF  .TFP  >  N. TBM. FLIGHT. PHASE 
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LET  . PK  =  0.0 
ELSE 

LET  .PK  =  DPT . PK ( . DWP,  . TFP,  . TTT) 
ALWAYS 

ELSE  ' ' KKV  failed  to  intercept 
LET  .PK  =  0.0 
ALWAYS 


' 'determine  result  of  shot 

'  '  *  Earl  stuff  * 


EARL. RESPONSE  =  RANDOM. F (RN . STREAM (. .AIR070 . 2 ) ) 

EB.P1  =  0.0 
EB.P2  =  0.0 
EB.P3  =  0.0 

CALL  EARL. STORE 
GIVEN 

. . AIR07  0 . 2  , 

. .RANDOM, 

EARL. RESPONSE, 

EB. PI, 

EB. P2, 

EB.  P3 

'  '  *  END  STUFF 
IF  .PK  >  0.0 

AND  EARL. RESPONSE  <=  . PK 
LET  .NUM. KILLED  =  1 
ELSE 

LET  .NUM. KILLED  =  0 
ALWAYS 


IF  (DT . PRINT . FLAG ( . . AIRF)  >=  . . LV3  AND 

DT . START . TIME ( . . AIRF)  <=  TIME.V  <=  DT . STOP . TIME (. .AIRF) ) 

OR  (DT. PRINT. FLAG (. .BPIPP)  >  . . LV0  AND 

DT. START. TIME (. .BPIPP)  <=  TIME.V  <=  DT . STOP . TIME (.. BPIPP) ) 
' 'Result  of  DTBM  engagement  with  a  TBM 
WRITE  TIME.V, 

TBU. ID ( . TBU)  , 

FG. ID (FL.FLT.GP ( .TBM.FLT)  )  , 

TA. ID ( .TBM.TAC)  , 

. PLATFORM. ID, 

DWP. ID ( .DWP)  , 

. TOTAL . ELAPSED . SECONDS , 

. INTERCEPT. FLAG, 

.PK, 

.NUM. KILLED 
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AS  D ( 8 , 4 ) ,  "  AIR070013  ",  I  5 ,  S  1 ,  I  7 ,  S  1 ,  I  5 ,  S  1 ,  I  7 ,  S  1 , 

S  1,  I  5,  S  1,  D  (6, 1)  ,  S  1,  I  1,  S  1,  D  (5, 3)  ,  S  1,  I  1,  / 

ALWAYS 

' ' expend  weapon 

IF  . DTBM . WPN . TYPE  =  .. SPACE . BASED . LASER 

SUBTRACT  1  FROM  SAT . SPACE . WPN . SHOTS . REMAINING (. DTBM . SAT ) 

IF  SAT. SPACE. WPN. SHOTS. REMAINING ( . DTBM. SAT)  <  0 
TRACE 
STOP 

OTHERWISE 

ELSE 

IF  DT. PRINT. FLAG ( . .DB.EXEC)  >=  . . LV1  AND 

DT. START. TIME ( . .DB.EXEC)  <=  TIME.V  <=  DT . STOP . TIME (.. DB . EXEC) 
CALL  DBS .220 .BEGIN. AIR. TO. AIR. ENGAGEMENT 
GIVEN 

. DTBM. X, 

. DTBM. Y 

ALWAYS 

SUBTRACT  1  FROM  AM . RNDS . REMAINING (. AM) 

LET  . TMI  =  TAM. INSTANCE ( .TAM) 

LET  .EXPENDED  =  TAM . NUMBER (. TAM) 

IF  TMI .CRITICAL. RESOURCE (TAM. INSTANCE ( .TAM) )  >  0 

SUBTRACT  .EXPENDED  FROM  TCR. LOADED (TMI . CRITICAL . RESOURCE (. TMI ) ) 
ADD  .EXPENDED  TO  TCR. EXPENDED (TMI . CRITICAL . RESOURCE (. TMI ) ) 
ALWAYS 

IF  DT. PRINT. FLAG ( . .DB.EXEC)  >=  . . LV1  AND 

DT. START. TIME (. .DB.EXEC)  <=  TIME.V  <=  DT . STOP . TIME (.. DB . EXEC) 
CALL  DBS. 222 .MUNITION. EXPEND. AIR. TO. AIR 
GIVEN 

. DTBM. FLT, 

.DTBM.FG, 

.TMI, 

.EXPENDED 

ALWAYS 

IF  DT. PRINT. FLAG (. .GRAPHER2K)  >=  . . LV1  AND 
DT. START. TIME (. .GRAPHER2K)  <=  TIME.V  <= 

DT . STOP .TIME ( . . GRAPHER2K) 

CALL  GRA. 730 .AIR. MUNITION. EXPENDITURE 
GIVEN 

. .AA.MUNT. LAUNCHED,  ''expenditure  type 
.DTBM. FLT, 

.TAM, 

1,  ' ' #  of  munt  sticks 

0,  ' 'not  used  here 

0,  ' 'not  used  here 

0,  ' 'not  used  here 

0  ' ' not  used  here 

ALWAYS 
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IF  DT. PRINT. FLAG (. .GRAPHER2K)  >=  . . LV1  AND 
DT. START. TIME (. .GRAPHER2K)  <=  TIME.V  <= 

DT . STOP .TIME ( . . GRAPHER2K) 

IF  .NUM. KILLED  >  0 

CALL  GRA. 740. AIR. TO. AIR. KILLS 
GIVEN 

.TBM.FLT,  ''victim  flight 

.DTBM.FLT,  ''killer  flight 

.NUM. KILLED  ''losses  to  victim  flight 

ALWAYS 

ALWAYS 

' 'set  up  DTBM  flight  group  statistics  for  air  reports 
IF  FG. MSN. STAT ( .DTBM. FG)  <>  0 
TRACE 
STOP 

OTHERWISE 

CREATE  MISSION. STATISTIC  CALLED  .MS 

LET  MS. FLIGHT. GROUP. ID (.MS)  =  FG . ID ( . DTBM . FG) 

LET  MS. TIME (.MS)  =  TIME.V 

LET  MS. TARGET. TYPE ( .MS)  =  .. INTERCEPT . POINT 
LET  MS. TARGET. ID (.MS)  =  FG . ID ( . TBM . FG) 

LET  MS. TARGET. NAME ( .MS)  =  "" 

LET  MS .KILLS ( .MS)  =  .NUM. KILLED 
LET  FG.MSN. STAT ( .DTBM. FG)  =  .MS 

CREATE  FLIGHT. STAT  CALLED  . FS 

LET  FLS. FLIGHT. ID(.FS)  =  FL . ID ( . DTBM. FLT) 

LET  FLS. TYPE. AC ( .FS)  =  FL . TYPE . AC (. DTBM . FLT ) 

LET  FLS. NUM. AC ( .FS)  =  FL . NUM . AC ( . DTBM . FLT ) 

LET  FLS. LOSSES ( .FS)  =  0 

FILE  .FS  IN  MS. FLIGHT. STAT. SET ( .MS) 

CREATE  TARGET. COMPONENT. STAT  CALLED  . TCS 

LET  TCS. TYPE. COMPONENT ( .TCS)  =  FL . TYPE . AC ( . TBM . FLT ) 

LET  TCS. COMPONENT. ID (.TCS)  =  FL . ID ( . TBM. FLT) 

LET  TCS . QTY . PCT . KILLED ( . TCS )  =  .NUM. KILLED 
FILE  .TCS  IN  MS. TARGET. COMPONENT. SET ( .MS) 

CREATE  AIR. MUNITION. STAT  CALLED  .AMS 
LET  AMS. TYPE ( .AMS)  =  .TAM 
LET  AMS. LAUNCHES ( .AMS)  =  1 

LET  AMS. TGT. TYPE. COMPONENT ( .AMS)  =  ..AIRCRAFT 

LET  AMS. TGT. COMPONENT. ID (.AMS)  =  FL . TYPE . AC ( . TBM . FLT ) 

FILE  .AMS  IN  FLS. MUNITION. SET ( .FS) 

IF  FG.MSN. STAT ( .TBM. FG)  =  0 

CREATE  A  MISSION. STATISTIC  CALLED  .MS 
LET  FG. MSN. STAT ( .TBM. FG)  =  .MS 
ELSE 

LET  .MS  =  FG.MSN. STAT ( .TBM. FG) 

ALWAYS 
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LET  MS. FLIGHT. GROUP. ID (.MS)  =  FG . ID ( . TBM . FG) 

LET  MS. TIME (.MS)  =  TIME.V 

LET  MS. TARGET. TYPE ( .MS)  =  .. INTERCEPT . POINT 
LET  MS. TARGET. ID (.MS)  =  FG . ID ( . DTBM . FG) 

LET  MS. TARGET. NAME ( .MS)  =  "" 

LET  MS .KILLS ( .MS)  =  0 

CREATE  FLIGHT. STAT  CALLED  . FS 

LET  FLS. FLIGHT. ID(.FS)  =  FL . ID ( . TBM. FLT) 

LET  FLS. TYPE. AC ( .FS)  =  FL . TYPE . AC ( . TBM . FLT ) 

LET  FLS.NUM.AC ( .FS)  =  FL . NUM . AC ( . TBM . FLT ) 

LET  FLS. LOSSES  (  .FS)  =  .NUM. KILLED 
FILE  .FS  IN  MS. FLIGHT. STAT. SET ( .MS) 

CREATE  TARGET. COMPONENT. STAT  CALLED  . TCS 

LET  TCS. TYPE. COMPONENT ( .TCS)  =  FL . TYPE . AC (. DTBM . FLT ) 

LET  TCS. COMPONENT. ID (.TCS)  =  FL . ID ( . DTBM. FLT) 

LET  TCS . QTY . PCT . KILLED ( . TCS )  =  0 

FILE  .TCS  IN  MS. TARGET. COMPONENT. SET ( .MS) 

IF  DT. PRINT. FLAG ( . .AA.ENC)  >=  . . LV1  AND 

DT. START. TIME ( . .AA.ENC)  <=  TIME.V  <=  DT . STOP . TIME (. .AA. ENC) 
CALL  UTL . 503 . LOCATE . SECTOR. FLOT . X 
GIVEN 

.TBM. LAUNCH. X, 

.TBM. LAUNCH. Y, 

. .BLUE 
YIELDING 
. FLOT . X 

LET  . BLUE. DIST. TO. FLOT  =  . TBM . LAUNCH . X  -  . FLOT . X 

CALL  UTL . 503 . LOCATE . SECTOR. FLOT . X 
GIVEN 

.TBM. LAUNCH. X, 

.TBM. LAUNCH. Y, 

.  .RED 
YIELDING 
. FLOT . X 

LET  .RED. DIST. TO. FLOT  =  . FLOT . X  -  . TBM . LAUNCH . X 

' 'Number  of  flights  in  the  air  engagement  and  the  distance 
' 'from  the  sector  FLOT  for  each  side 
WRITE  TIME.V, 

1 ,  ' ' blue  flight 

.BLUE. DIST. TO. FLOT, 

1 ,  ' ' red  flight 

.RED. DIST. TO. FLOT 

AS  D ( 8 , 4 ) , "  AIR070091  ",I  3,S  1,1  9,S  1,1  3,S  1,1  9,  / 


' 'Describes  a  DTBM  flight  in  an  air  engagement. 
WRITE  TIME.V, 

TA. USED. INDEX(FL. TYPE. AC ( . DTBM. FLT)  )  , 
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FL. MISS ION ( .DTBM.FLT)  , 

FL . NUM . AC ( . DTBM . FLT ) , 

FG. STATUS (FL.FLT.GP ( .DTBM.FLT) ) 

AS  D  ( 8 , 4 ) , "  AIR070101  ",I  5,S  1,1  5,S  1,1  5,S  1,1  5,  / 


' 'Describes  a  TBM  flight  in  an  air  engagement. 

WRITE  TIME . V, 

TA. USED. INDEX (FL. TYPE. AC ( .TBM. FLT)  )  , 

FL. MISS ION ( .TBM. FLT)  , 

FL . NUM . AC ( . TBM . FLT ) , 

FG. STATUS (FL.FLT.GP ( .TBM. FLT) ) 

AS  D  ( 8 , 4 ) , "  AIR070101  ",I  5,S  1,1  5,S  1,1  5,S  1,1  5,  / 

ALWAYS 

IF  DT. PRINT. FLAG( . .AA. KILLS)  >=  . . LV1  AND 

DT. START. TIME (. .AA. KILLS)  <=  TIME.V  <=  DT . STOP . TIME (.. AA . KILLS ) 
' 'Air-to-Air  kills  for  post  processor 
WRITE  TIME.V, 

TA. USED. INDEX(FL. TYPE. AC ( . DTBM. FLT)  )  , 

FL. MISS ION ( .DTBM.FLT)  , 

FL . NUM . AC ( . DTBM . FLT ) , 

TA. USED. INDEX(FL. TYPE. AC ( . TBM. FLT) ) , 

FL. MISS ION ( .TBM. FLT) , 

FL . NUM . AC ( . TBM . FLT ) , 

0  ' 'nun  lost  by  DTBM 

AS  D ( 8 , 4 ) ,  "  AIR070081" ,  S  1 ,  I  5 ,  S  1 ,  I  5 ,  S  1 , 

I  5,  S  1,  I  5,  S  1,  I  5,  S  1,  I  5,  S  1,  I  5,  / 

' 'Air-to-Air  kills  for  post  processor 
WRITE  TIME.V, 

TA. USED. INDEX(FL. TYPE. AC ( . TBM. FLT)  )  , 

FL. MISS ION ( .TBM. FLT)  , 

FL . NUM . AC ( . TBM . FLT ) , 

TA. USED. INDEX (FL. TYPE. AC ( .DTBM.FLT) ) , 

FL. MISS ION ( .DTBM.FLT)  , 

FL . NUM . AC ( . DTBM . FLT ) , 

.NUM. KILLED 

AS  D ( 8 , 4 ) ,  "  AIR070081" ,  S  1 ,  I  5 ,  S  1 ,  I  5 ,  S  1 , 

I  5,  S  1,  I  5,  S  1,  I  5,  S  1,  I  5,  S  1,  I  5,  / 

ALWAYS 

' 'report  results 

CALL  UTL. 270. MISSION. STAT. REPORT 
GIVEN 

. DTBM. FG, 

. .DTBM 

CALL  UTL. 270. MISSION. STAT. REPORT 
GIVEN 

. TBM. FG, 

. .ALL. FLIGHTS 

ALWAYS 


IF  .NUM. KILLED  =  1 
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IF  . DTBM . WPN . TYPE  <>  .. SPACE . BASED . LASER 

IF  DT. PRINT. FLAG ( . .DB.EXEC)  >=  . . LV1  AND 
DT. START. TIME ( . .DB.EXEC)  <=  TIME.V  <= 

DT. STOP. TIME ( . .DB.EXEC) 

CALL  DBS. 221. FLT. LOSS. AIR. TO. AIR 
GIVEN 

.TBM.FLT, 

.TBM.FG, 

1 ,  ' ' loss 

..YES,  ''before  target 

TA. ID ( .TBM.TAC) , 

FL. MISS ION ( .TBM.FLT) 

ALWAYS 

IF  (POST. PROCESS. GRAPHICS  =  ..MAP  OR 

POST. PROCESS. GRAPHICS  =  ..BOTH)  AND 
DT. PRINT. FLAG (. .GRAA)  GE  . . LV2  AND 

DT. START. TIME (. .GRAA)  LE  TIME.V  LE  DT . STOP . TIME (.. GRAA) 

IF  FG. SIDE ( .DTBM. FG)  =  ..BLUE 
LET  . BLUE . FG  =  . DTBM. FG 
LET  . RED . FG  =  .TBM.FG 
ELSE 

LET  . BLUE . FG  =  .TBM.FG 
LET  . RED . FG  =  . DTBM. FG 
ALWAYS 

' ' FOR  NOW 


LET 

. ENGAGE. X  = 

.5  *  ( .TBM. LAUNCH. X  + 

.TBM.TGT.X) 

LET 

. ENGAGE. Y  = 

.5  *  ( .TBM. LAUNCH. Y  + 

.TBM.TGT.Y) 

LET 

. SAVE. TBM. X 

=  FG. X. COORD ( .TBM.FG) 

LET 

. SAVE. TBM. Y 

=  FG. Y. COORD ( .TBM.FG) 

LET  FG. X. COORD ( .TBM.FG)  =  . ENGAGE . X 
LET  FG. Y. COORD ( .TBM.FG)  =  . ENGAGE . Y 

IF  DT. PRINT. FLAG (. .GRAA)  >=  . . LV3 

LET  FG.CURR. PATH (.TBM.FG)  =  F . FG . PATH . SET (. TBM . FG) 

CALL  GRA. 600 . PRINT . MISSION . MOVE . EVENT 
GIVEN 

.TBM.FG, 

.TBM.FLT 

LET  FG.CURR. PATH (.TBM.FG)  =  0 
ALWAYS 

LET  FL. NUM. AC. LOST. LAST. ATTACK ( .TBM.FLT)  =  .NUM. KILLED 
LET  FL. ENGAGE. FLAG ( .TBM.FLT)  =  ..YES 
LET  FL. ENGAGE. FLAG ( .DTBM. FLT)  =  ..YES 

''TEMPORARILY  MOVE  DTBM  FLIGHT  GROUP  SO  GRAPHICS  WILL  SHOW 
' '  ENGAGEMENT  ALONG  TBM  FLIGHT  PATH 
LET  .SAVE. DTBM. X  =  FG . X . COORD (. DTBM . FG) 

LET  .SAVE. DTBM. Y  =  FG . Y . COORD (. DTBM . FG) 
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LET  FG. X. COORD ( .DTBM.FG)  =  . ENGAGE . X 
LET  FG. Y. COORD ( .DTBM.FG)  =  . ENGAGE . Y 

CALL  GRA. 540. AIR. TO. AIR. LOSS 
GIVING 

.BLUE . FG, 

. RED . FG 

LET  FG. X. COORD ( .DTBM.FG)  =  . SAVE . DTBM. X 
LET  FG. Y. COORD ( .DTBM.FG)  =  .SAVE. DTBM. Y 
LET  FG. X. COORD ( .TBM.FG)  =  .SAVE.TBM.X 
LET  FG. Y. COORD ( .TBM.FG)  =  . SAVE . TBM. Y 

LET  FL. NUM. AC. LOST. LAST. ATTACK ( .TBM. FLT)  =  0 
LET  FL. ENGAGE. FLAG ( .TBM. FLT)  =  ..NO 
LET  FL. ENGAGE. FLAG ( .DTBM. FLT)  =  ..NO 
ALWAYS 
ALWAYS 

' 'clean  up  tbm  flight  &  flight  group 
CALL  UTL. 090. DESTROY. AIRCRAFT 
GIVEN 

.TBM. FLT, 

.NUM. KILLED, 

. .AIRCRAFT 


CALL  AIR. 510. DECISION. POINT 
GIVEN 

.TBM.FG, 

..YES  ''aircraft  lost 

YIELDING 

.CONTINUE 

IF  .CONTINUE  <>  ..NO 
TRACE 
STOP 

OTHERWISE 

FOR  EVERY  . EVENT4  IN  EV . S ( I . AIR . 004 . MOVE . FLT . GP) 
WITH  MFG.FLT.GP ( .EVENT4)  =  .TBM.FG 

FIND  THE  FIRST  CASE 

IF  NONE 
TRACE 
STOP 

OTHERWISE 

CANCEL  AIR. 004. MOVE. FLT. GP  CALLED  . EVENT4 

DESTROY  AIR. 004. MOVE. FLT. GP  CALLED  . EVENT4 
ALWAYS 


IF  . DTBM. WPN. TYPE  <>  .. SPACE . BASED . LASER 
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IF  DT. PRINT. FLAG ( . .DB.EXEC)  >=  . . LV1  AND 

DT. START. TIME ( . .DB.EXEC)  <=  TIME.V  <=  DT . STOP . TIME (.. DB . EXEC) 
CALL  DBS . 2 2 9 . END . AIR . TO . AIR . ENGAGEMENT 
ALWAYS 

' 'continuation  decision  of  no  will  cause  DTBM  to  leave  station 
CALL  AIR. 510. DECISION. POINT 
GIVEN 

.DTBM.FG, 

. .NO  ' 'aircraft  lost 

YIELDING 

.CONTINUE 

ALWAYS 

EXITROUTINE 

ENDROUTINE 
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Appendix  G:  MRG32k3a  Routines  and  Variables  (18) 


Routines: 

Rng Stream  (const  char  *name  = 

This  constructor  creates  a  new  stream  with  (optional)  descriptor  name.  It  initializes  its 
seed  Ig,  and  sets  Bg  and  Cg  to  Ig.  It  also  sets  its  anti  and  incPrec  switches  to  false. 
The  seed  Ig  is  equal  to  the  initial  seed  of  the  package  if  this  is  the  first  stream  created; 
otherwise  it  is  Z  steps  ahead  of  the  seed  of  the  most  recently  created  stream. 

void  SetPackageSeed  (const  unsigned  long  seed[6]); 

Sets  the  initial  seed  sO  of  the  package  to  the  six  integers  in  the  vector  seed.  The  first  3 
integers  in  the  seed  must  all  be  less  than  ml  =  4294967087,  and  not  all  0;  and  the  last 
3  integers  must  all  be  less  than  m2  =  4294944443,  and  not  all  0.  If  this  method  is  not 
called,  the  default  initial  seed  is  (12345,  12345,  12345,  12345,  12345,  12345). 

void  ResetStartStream  (); 

Reinitializes  the  stream  to  its  initial  state:  Cg  and  Bg  are  set  to  Ig. 

void  ResetStartSubstream  (); 

Reinitializes  the  stream  to  the  beginning  of  its  current  substream:  Cg  is  set  to  Bg. 

void  ResetNextSubstream  (); 

Reinitializes  the  stream  to  the  beginning  of  its  next  substream:  Ng  is  computed,  and 
Cg  and  Bg  are  set  to  Ng. 

void  SetAntithetic  (bool  a); 

If  a  =  true,  the  stream  will  start  generating  antithetic  variates,  i.e.,  1  -  U  instead  of  U, 
until  this  method  is  called  again  with  a  =  false. 

void  Increased  Precis  (bool  incp); 

After  calling  this  method  with  incp  =  true,  each  call  to  the  generator  (direct  or 
indirect)  for  this  stream  will  return  a  uniform  random  number  with  more  bits  of 
resolution  (53  bits  if  machine  follows  IEEE  754  standard)  instead  of  32  bits,  and  will 
advance  the  state  of  the  stream  by  2  steps  instead  of  1 .  More  precisely,  if  s  is  a  stream 
of  the  class  RngStream,  in  the  nonantithetic  case,  the  instruction  “u  =  s.RandUOl  ()” 
will  be  equivalent  to  “u  =  (s.RandUOl  ()  +  s.RandUOl  ()  *  fact)  %  1.0”  where  the 
constant  fact  is  equal  to  2-24.  This  also  applies  when  calling  RandUOl  indirectly 
(e.g.,  via  Randlnt,  etc.).  By  default,  or  if  this  method  is  called  again  with  incp  =  false, 
each  call  to  RandUOl  for  this  stream  advances  the  state  by  1  step  and  returns  a 
number  with  32  bits  of  resolution. 

void  SetSeed  (const  unsigned  long  seed[6]); 

Sets  the  initial  seed  Ig  of  the  stream  to  the  vector  seed.  The  vector  seed  should  contain 
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valid  seed  values  as  described  in  SetPackageSeed.  The  state  of  the  stream  is  then 
reset  to  this  initial  seed.  The  states  and  seeds  of  the  other  streams  are  not  modified.  As 
a  result,  after  calling  this  method,  the  initial  seeds  of  the  streams  are  no  longer  spaced 
Z  values  apart.  We  discourage  the  use  of  this  method;  proper  use  of  the  Reset* 
methods  is  preferable. 

void  AdvanceState  (long  e,  long  c); 

Advances  the  state  by  n  steps  (see  below  for  the  meaning  of  n),  without  modifying  the 
states  of  other  streams  or  the  values  of  Bg  and  Ig  in  the  current  object.  If  e  >  0,  then  n 
=  2e  +  c;  if  e  <  0,  then  n  =  -2-e  +c;  and  if  e  =  0,  then  n  =  c.  Note:  c  is  allowed  to  take 
negative  values.  We  discourage  the  use  of  this  method. 

void  GetState  (unsigned  long  seed[6])  const; 

Returns  in  seed[0..5]  the  current  state  Cg  of  this  stream.  This  is  convenient  if  we  want 
to  save  the  state  for  subsequent  use. 

void  WriteState  ()  const; 

Writes  (to  standard  output)  the  current  state  Cg  of  this  stream, 
void  WriteStateFull  ()  const; 

Writes  (to  standard  output)  the  value  of  all  the  internal  variables  of  this  stream:  name, 
anti,  incPrec,  Ig,  Bg,  Cg. 

double  RandUOl  (); 

Normally,  returns  a  (pseudo)random  number  from  the  unifonn  distribution  over  the 
interval  (0,  1),  after  advancing  the  state  by  one  step.  The  returned  number  has  32  bits 
of  precision  in  the  sense  that  it  is  always  a  multiple  of  1/(232-208).  However,  if 
IncreasedPrecis(true)  has  been  called  for  this  stream,  the  state  is  advanced  by  two 
steps  and  the  returned  number  has  bits  of  precision. 

long  Randlnt  (long  i,  long  j); 

Returns  a  (pseudo)random  number  from  the  discrete  uniform  distribution  over  the 
integers  {i,  i  +  1, .  . . ,  j}.  Makes  one  call  to  RandUOl . 


Variables: 

double  Cg[6],  Bg[6],  Ig[6]; 

Vectors  to  store  the  current  seed,  the  beginning  of  the  current  block  (substream)  and 
the  beginning  of  the  current  stream. 

bool  anti,  incPrec; 

Variables  to  indicate  whether  to  generate  antithetic  or  increased  precision  random 
numbers. 
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std::  string  name; 

String  to  store  the  optional  name  of  the  current  Rng Stream  object, 
static  double  nextSeed[6]; 

Static  vector  to  store  the  beginning  state  of  the  next  Rng  Stream  to  be  created 
(instantiated). 

double  U01  (); 

The  backbone  unifonn  random  number  generator, 
double  UOld  (); 

The  backbone  unifonn  random  number  generator  with  increased  precision. 
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Appendix  H:  CHARACTERISTICS  OF  VRTs  (6) 
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CHARACTERISTICS  OF  VARIANCE  REDUCTION  TECHNIQUES  (Continued) 
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Appendix  I:  SQUADRON.DAT  File  Example  (33) 


Squadrons  (squadron.dat) 

SQUADRONS .305 

NUMBER. OF. MISSION. CLASSES:  12 

(N.MISSION. CLASS) 

AIR. SUPERIORITY 

DEEP . STRIKE 

GROUND  .  SUPPORT 

JAMMER 

MULTI .ROLE 

RECCE 

WEASEL 

AWACS 

JSTARS 

TANKER 

TBM 

BP  I 

(Mission  class  names  (MC.NAME)) 

NUMBER. OF. SORTIE. PROFILES:  14 

(N.SORTIE.RATE. PROFILE) 

1001  "A-10" 

(Sortie  rate  profile  ID  (SRP.ID)  and  Name  (SRP.NAME)) 

DAY . IN . THEATER . . AUTH . QTY . SORT/DAY . . AC . MAX . SORT/DAY 
1.00  4.00  4.00 

(Start  time  from  arrival  in  theater  (SRE.START.TIME),  Nominal  sortie  rate 
(SRE.MAX.RATE),  Maximum  sortie  rate  (SRE.MAX.RATE)) 

END . PROFILE 

1002  "F-16" 

DAY . IN . THEATER . . AUTH . QTY . SORT/DAY . . AC . MAX . SORT/DAY 
1.00  3.60  6.00 

6.00  2.50  5.00 

END . PROFILE 


NUMBER . OF . SQUADRONS :  3  0 

(Number  of  squandrons  (will  be  added  to  N. TBM. UNIT  to  form  N. SQUADRON)) 

1001  "1ST  FTR" 

(Squadron  ID  (SQN.ID)  and  Name  (SQN.NAME)) 

SIDE . . SUP . CMD . ID . . TYPE . AC . ID . . AUTH . QTY . . AR . PRIORITY . . . RECCE . PRIORITY 
1  1102  1005  24  0  0 

(Side  the  squadron  belongs  to  (1=BLUE,  2=RED)  (SQN.SIDE),  Superior  command  ID 
(SQN. COMMAND),  Aircraft  type  ID  (SQN.TYPE  AC),  Quantity  of  aircraft  (SQN.QTY.AC), 

Priority  for  air  refueling  (must  be  at  least  0  and  at  most  100)  (SQN.REFUEL.PRIORITY), 

Squadron  reconnaissance  priority  (SQN.RECCE.PRIORITY)) 

MOB . ID . . DISP . AB . ID . . SERV . KIT . ID . . SORT . PROF . ID . . MISSION . CLASS 
1002  1005  1005  1005  AIR .  SUPERIORITY 

(Main  operating  base  ID  number  (SQN. MOB),  Dispersal  base  ID  number  (SQN. DISPERSAL. BASE), 

Service  Kit  ID  number  (SQN. SERVICE. CONFIG),  Sortie  profile  ID  number 

(SQN. SORTIE. RATE.PROFILE),  Squadron  mission  class  (SQN.MISSION.CLASS)) 


1.1 


.  . DCA .  . ODCA .  . HVAA .  . BARC .  . FSWP .  . EAIR .  .  . STI .  .  . CAS .  .  . BA I .  .  . INT .  .  . OCA .  . OTBM .  . DTBM 
100  100  100  100  100  100  0  0  0  0  0  0  0 
.  DSED .  .SSUP.  .CSUP.  .ESUP.  .SJAM.  .CJAM.  .EJAM.  . RECC .  . SREC .  .  . AEW .  .  .AAR.  .LIFT.  . XXXX .  .RESV 
00  00000  000000  100 


(Assigned  squadron  effectiveness  for  each  type  mission  (percent) 
(MSQ.ASSIGNED.EFFECTIVENESS))’ 


ORDERS 

1.2  FOB  1005  0.25  UNIF  1.0  1.5 

1.2  SORTIE. PROFILE  1007 
END . ORDERS 


1002  " 1ST  FTR-BMBR " 

SIDE . . SUP . CMD . ID . . TYPE . AC . ID . . AUTH . QTY . . AR . PRIORITY . . . RECCE . PRIORITY 
1  1102  1002  24  0  0 

MOB. ID. .DISP.AB.ID. . SERV . KIT . ID . . SORT . PROF . ID . . MISSION . CLASS 
1002  1005  1002  1002  DEEP .  STRIKE 

. . DCA . . ODCA . . HVAA . . BARC . . FSWP . . EAIR . . . STI . . . CAS . . . BAI . . . INT . . . OCA . . OTBM . . DTBM 
100  100  100  100  100  100  20  100  100  100  100  100  100 
.DSED. .SSUP. .CSUP. .ESUP. .SJAM. .CJAM. .EJAM. .RECC. .SREC. . .AEW. . .AAR. .LIFT. .XXXX. .RESV 
0  0  0  0  0  0  0  100  0  0  0  0  0  100 


ORDERS 

1.2  MSN.  CLASS  MULT I.  ROLE  2 
END . ORDERS 


END . SQUADRONS 
Notes 

1 .  The  squadron  mission  effectiveness  is  multiplied  by  the  type  aircraft  mission  effectiveness  to  form  the  mission 
effectiveness  used  in  the  LP  that  allocates  sorties  to  missions. 

2.  Squadron  Orders  Format 

Time  Order  <Argl><Arg2>  where  Time  =  Time  to  execute  order 


Order 

Arguments 

DOB 

<air  base  ID> 

MOB 

<air  base  ID> 

FOB 

<air  base  IDxprob  return  to  MOB> 
<return  to  MOB  delay  time  function> 

SORTIE. PROFILE 

<sortie  profile  ID> 

MSN.EFF 

<mission><effectiveness> 

SUPERIOR 

<air  planning  command  ID> 

MERGE 

<squadron  ID> 

AC.QTY 

<change  in  aircraft  quantity> 

MSN.QTY 

ARRIVE 

emission  class> 

KIT.ARRIVE 

<kit,ID> 

AR.PRIORITY 

<0-1 00> 

Effect 

change  dispersal  base 

change  main  operating  base 

change  squadron  operations  to  a  forward 

operating  base 

change  sortie  profiles 

change  squadron  mission  effectiveness 

change  superior  command  of  the  squadron 
merge  with  another  squadron  of  the  same 

type  AC 

add/subtract  aircraft  inventory 
change  mission  class 
arrive  in  theater 

provides  squadron  with  this  type  of  service 

kit 

changes  in  air  refueling  priority 
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